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Introduction  to  ARL 

The  Army  Research  Laboratory  of  the  U.S.  Army  Research,  Development  and  Engineering  Command  (RDECOM)  is  the  Army's 
corporate  laboratory.  ARL's  research  continuum  focuses  on  basic  and  applied  research  (6.1  and  6.2)  and  survivability/lethality 
and  human  factors  analysis  (6.6).  ARL  also  applies  the  extensive  research  and  analysis  tools  developed  in  its  direct  mission 
program  to  support  ongoing  development  and  acquisition  programs  in  the  Army  Research,  Development  and  Engineering 
Centers  (RDECs),  Program  Executive  Offices  (PEOs)/Program  Manager  (PM)  Offices,  and  Industry.  ARL  has  consistently  provided 
the  enabling  technologies  in  many  of  the  Army's  most  important  weapons  systems. 

The  Soldiers  of  today  and  tomorrow  depend  on  us  to  deliver  the  scientific  discoveries,  technological  advances,  and  the  analyses 
that  provide  Warfighters  with  the  capabilities  to  execute  full-spectrum  operations.  ARL  has  Collaborative  Technology  Alliances  in 
Micro  Autonomous  Systems  and  Technology,  Robotics,  Cognition  and  Neuroergonomics,  and  Network  Science,  an  International 
Technology  Alliance,  and  new  Collaborative  Research  Alliances  in  Multiscale  Multidisciplinary  Modeling  of  Electronic  Materials 
and  Materials  in  Extreme  Dynamic  Environments.  ARL's  diverse  assortment  of  unique  facilities  and  dedicated  workforce  of 
government  and  private  sector  partners  make  up  the  largest  source  of  world  class  integrated  research  and  analysis  in  the  Army. 

ARL  Mission 

The  mission  of  ARL  is  to  provide  innovative  science,  technology,  and  analyses  to  enable  full  spectrum  operations. 
Our  Vision 

America's  Laboratory  for  the  Army:  Many  Minds,  Many  Capabilities,  Single  Focus  on  the  Soldier 
ARL's  Organization 

•Army  Research  Office  (ARO)  -  Initiates  the  scientific  and  far  reaching  technological  discoveries  in  extramural  organizations: 
educational  institutions,  nonprofit  organizations,  and  private  industry. 

•Computational  and  Information  Sciences  Directorate  -  Scientific  research  and  technology  focused  on  information  processing, 
network  and  communication  sciences,  information  assurance,  battlespace  environments,  and  advanced  computing  that 
create,  exploit,  and  harvest  innovative  technologies  to  enable  knowledge  superiority  for  the  Warfighter. 

•Human  Research  and  Engineering  Directorate  -  Scientific  research  and  technology  directed  toward  optimizing  Soldier 
performance  and  Soldier-machine  interactions  to  maximize  battlefield  effectiveness  and  to  ensure  that  Soldier  performance 
requirements  are  adequately  considered  in  technology  development  and  system  design. 

•Sensors  and  Electron  Devices  Directorate  -  Scientific  research  and  technology  in  electro-optic  smart  sensors,  multifunction 
radio  frequency  (RF),  autonomous  sensing,  power  and  energy,  and  signature  management  for  reconnaissance,  intelligence, 
surveillance,  target  acquisition  (RISTA),  fire  control,  guidance,  fuzing,  survivability,  mobility,  and  lethality. 

•Survivability/Lethality  Analysis  Directorate  -  Integrated  survivability  and  lethality  analysis  of  Army  systems  and  technologies 
across  the  full  spectrum  of  battlefield  threats  and  environments  as  well  as  analysis  tools,  techniques,  and  methodologies. 

•Vehicle  Technology  Directorate  -  Scientific  research  and  technology  addressing  propulsion,  transmission,  aeromechanics, 
structural  engineering,  and  robotics  technologies  for  both  air  and  ground  vehicles. 

•Weapons  and  Materials  Research  Directorate  -  Scientific  research  and  technology  in  the  areas  of  weapons,  protection,  and 
materials  to  enhance  the  lethality  and  survivability  of  the  Nation's  ground  forces. 

ARL  Workforce  in  2013 

•  1,980  Civilians  -  38  Military 

•  1080  Contractors  (1027  full-time/53  part-time) 
•1,379  Research  Performing  Workforce 

•  552  (40%)  hold  PhDs 

•  11  STs/23  ARL  Fellows 

ARLs  Primary  Sites 

•  Aberdeen  Proving  Ground,  MD 

•  Adelphi  Laboratory  Center,  MD 

•  White  Sands  Missile  Range,  NM 

•  Raleigh-Durham,  NO 

•  Orlando,  FL 


Unique  ARL  laboratory  facilities  and  modeling  capabilities  provide  our  scientists  and 
engineers  with  a  world-class  research  environment. 


Visit  ARLs  web  site  at  www.arl.army.mil 


FOREWORD 


Welcome  to  our  most  recent  edition  of  the  Research@ARL  monograph  series, 
a  compilation  of  recently  published  journal  articles  highlighting  the  US  Army 
Research  Laboratory  (ARL)  research  programs  in  autonomous  systems.  ARL 
is  a  robust  research  laboratory  with  a  staff  committed  to  promoting  diversity 
of  opinion,  strategy,  and  problem  solving  among  a  variety  of  specialized 
disciplines.  ARL  capitalizes  on  our  expertise  in  basic  and  applied  research 
across  multiple  science  and  engineering  disciplines,  while  simultaneously 
drawing  upon  the  talents  of  others,  nationally  and  internationally,  through 
collaborative  alliances  with  industry,  laboratories,  and  academia. 

ARL  has  a  broad  interest  in  the  basic  and  applied  research  associated  with 
the  understanding,  creation  and  development  of  intelligent  systems  to 
assist  and  augment  the  Army's  current  and  future  mission.  Our  long-term 
commitment  to  interdisciplinary  scientific  research  for  intelligent  systems 
strongly  positions  our  staff  to  discover,  innovate,  and  transition  knowledge  and  technology  for  the  nation's  benefit. 
Our  discoveries  enhance  our  country's  technology  base,  our  national  security,  and,  ultimately,  the  effectiveness 
and  safety  of  our  Soldiers. 

Current  robotics  capabilities  are  primarily  controlled  by  remote  operators,  and  have  been  showcased  in  recent 
military  conflicts  for  applications  such  as  aerial  reconnaissance,  searching  caves,  and  disposing  of  unexploded 
ordinance.  The  increased  use  of  autonomous  platforms  was  recommended  to  the  Department  of  Defense  in  a  2012 
Defense  Science  Board  report.  We  understand  significant  advances  will  be  required  to  enable  controlled  movement 
with  apparent  intent.  This  volume  of  Research@ARL  highlights  our  advances  in  mobility  of  millimeter-scale  robotic 
platforms,  intelligent  robotic  control,  and  supervisory  control  of  robotic  systems  with  enhanced  human-machine 
collaboration.  ARL  is  actively  expanding  the  level  of  autonomous  intent  in  engineered  robotic  systems  through  the 
combined  investigation  of  materials,  mechanics,  electronics,  signal  processing,  and  technology  integration.  By 
increasing  the  capability  for  autonomous  action  by  a  single  platform,  ARL  seeks  to  create  mixed  teams  of  humans 
and  robotic  platforms  through  research  on  human  behavior. 

As  the  new  director  of  ARL,  I  am  pleased  to  share  this  Research@ARL  volume.  We  are  committed  to  discover, 
innovate,  and  transition  science  and  engineering  for  the  next  Army  and  the  future  Army. 


Dr.  Thomas  P.  Russell 

Director,  U.S.  Army  Research  Laboratory 
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Introduction  to  ARL  Research  in  Autonomous  Systems  and  Teclinoiogy  Researcli 
J.  N.  IVIait,  B.  IVI.  Sadier,  A.  Swami,  and  B.  J.  West 

1.  Introduction 

Unlike  the  apocalyptic  scenarios  envisioned  in  the  movies,  autonomous  systems  perform  critically  helpful  roles  for  today's 
Soldiers  and  first  responders.  Routinely  aerial  platforms  surveil  large  areas  of  urban  terrain  without  an  on-board  pilot. 
Ground  platforms  neutralize  improvised  explosive  devices  on  streets  in  Baghdad  and  investigate  the  radiated  environment 
of  the  disabled  Fukushima  Nuclear  Plant.  For  over  a  decade,  the  U.S.  Army  Research  Laboratory  (ARL)  has  contributed 
significantly  to  the  development  of  technologies  capable  of  enabling  autonomous  behavior  by  mobile  platforms.  This 
volume  presents  some  of  those  contributions  and  this  introduction  provides  context  for  the  research  performed  at  ARL. 

Human  fascination  with  animating  the  inanimate  is  as  old  as  the  Golem  from  Jewish  folklore.  The  Golem  is  a  creature  of 
mud  that  can  be  animated  with  magic  words,  but  it  is  not  intelligent  —  when  commanded  to  perform  a  task,  it  performs 
its  instructions  literally,  without  nuance  or  understanding.  In  fact,  a  modern  version  of  the  Golem  legend  gave  us  the 
word  robot.  Karel  Capek,  a  Czech  playwright,  introduced  and  popularized  the  word  robot  (from  the  Czech  word  for  labor, 
"robota")  in  his  1921  science  fiction  play  "Rossum's  Universal  Robots." 

Since  that  time,  the  entertainment  industry  has  created  a  wide  variety  of  robotic  platforms,  in  various  shapes  and  sizes 
with  various  levels  of  intelligence^.  Hollywood's  terrestrial  robots  range  from  R2D2  and  Wall-E  to  the  anthropomorphic 
C3P0  and  the  Terminator,  and  provide  us  with  a  visual  taxonomy  for  characterizing  platforms.  The  defining  features  for 
robotic  platforms  are  their  method  for  mobility,  their  suite  of  sensors,  the  manner  and  degree  to  which  they  interact 
with  and  manipulate  their  environment,  the  degree  to  which  they  collaborate  with  humans,  the  degree  to  which  they 
collaborate  with  other  robotic  platforms,  their  level  of  intelligence,  and  the  degree  to  which  humans  influence  or  control 
their  actions  and  the  manner  in  which  they  do  so.  Although  designers  take  considerable  inspiration  from  biology,  they 
produce  a  wide  variety  of  platforms  that  differ  considerably  from  anything  found  in  nature.  Through  engineering  as 
opposed  to  evolution,  designers  make  trade-offs  that  nature  does  not. 

As  we  discuss  in  Sec.  2  of  this  introduction,  most  military  applications  do  not  require  anthropomorphic  robots.  We 
describe  in  Section  3  how  ARL  executes  research  on  autonomous  systems  through  its  Robotics  Enterprise.  In  Sec.  4,  we 
discuss  the  technical  issues  faced  by  designers  of  autonomous  systems  and  describe  some  of  the  work  ARL  researchers 
have  performed  to  overcome  them.  We  close  in  Sec.  5  with  a  discussion  of  nontechnical  issues  related  to  the  military 
use  of  autonomous  systems. 

2.  Army  Applications 

"Dirty,  dangerous,  and  dull"  is  the  oft-repeated  description  of  applications  best  suited  to  robotic  platforms.  Inaccessible 
has  also  been  added  to  the  list,  albeit  without  an  appropriately  alliterative  synonym.  Army  applications  that  fall  under 
this  rubric  include  explosive  ordinance  disposal,  logistics  (in  effect,  replacing  the  Army  mule  with  its  robotic  equivalent), 
and  situational  awareness,  which  includes  perimeter  security,  building  clearing,  and  interrogating  confined  spaces  in 
buildings  or  rubble  that  are  too  small  for  humans.  Further,  in  addition  to  use  as  remote  sensing  platforms,  the  Army  has 
considered  using  robots  for  remote  fires  and  for  remote  communications.  It  has  also  considered  using  robots  to  protect, 
treat,  and  remove  the  wounded  from  the  battlefield. 

However,  most  robots  presently  operating  in  military  theaters  are  incapable  of  performing  these  functions  autonomously. 
They  are  typically  large,  tele-operated  platforms  whose  behavior  is  controlled  remotely  by  a  human;  sometimes,  as  in  the 
case  of  the  Predator  unmanned  aerial  vehicle,  half  a  world  away. 

Communication  limits  the  performance  of  tele-operated  systems.  For  example,  bandwidth  constraints  limited  the  number 
of  unmanned  aerial  vehicles  that  could  fly  simultaneously  in  Afghanistan  to  no  more  than  four.  In  addition,  normal 
communication  delays  (latency)  limit  operational  tempo.  More  limiting,  tele-operation  requires  at  least  one  dedicated 
pair  of  eyes  and  hands.  In  combat  operations,  this  implies  the  operator  requires  a  protector,  which  is  a  disadvantageous 
2:1  ratio  of  humans  to  robots. 

To  overcome  the  limitations  of  tele-operation  and  advance  capabilities  beyond  the  current  state  of  supervised  autonomy, 
ARL  seeks  to  enable  the  integration  of  robotic  platforms  into  a  team  of  Soldiers.  To  act  as  teammates,  robotic  systems 
need  to  perform  their  mission  effectively.  They  must  operate  reliably  within  clearly  defined  operational  bounds  and 
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Figure  1.  The  Last  100  Meters,  (a)  Mounted  and  dismounted  soidiers  aiong  witli  autonomous  veliides  approach 
an  urban  structure,  (b)  Bacl<pacl<-sized  ground  piatforms  reiease  paim-sized  ground  and  air  piatforms.  Paim-sized 
piatforms  (c)  approach  and  (d)  enter  the  structure,  and  (e)  reiay  information  to  dismounts. 

engender  the  trust  of  their  human  teammates.  At  a  technical  level,  they  need  to  move  in  a  tactically  beneficial  way, 
observe  salient  events  in  their  environment,  and  communicate  efficiently.  Rather  than  the  Army's  classic  dictum  "move, 
shoot,  and  communicate",  ARL's  research  focus  is  on  autonomous  capabilities  to  "move,  sense,  and  communicate." 

The  "Last  100  Meters"  vignette  represented  in  Fig.  1  reflects  some  of  the  capabilities  ARL  is  pursuing.  In  this  vignette, 
mounted  and  dismounted  Soldiers  approach  an  urban  objective  supported  by  large  autonomous  ground  and  aerial 
platforms.  Small  ground  and  aerial  platforms  provide  an  initial  assessment  of  the  structure,  e.g.,  they  identify  areas  of 
activity  and  potential  points  of  ingress.  The  small  platforms  then  release  even  smaller  platforms,  which  enter  the  building 
and  provide  more  situational  awareness.  For  example,  they  generate  maps,  identify  and  label  potential  threats,  and  plan 
paths  for  dismounts,  all  of  which  the  network  of  autonomous  platforms  conveys  to  troops  readying  themselves  to  enter 
the  structure. 

3.  The  ARL  Enterprise 

To  address  the  technical  challenges  presented  by  our  vignette,  ARL  conducts  research  internally  and  collaboratively 
with  industry  and  academia  through  a  single  investigator  program  supported  by  the  Army  Research  Office,  through  two 
collaborative  technology  alliances  (CTAs),  the  Robotics  CTA^  and  the  Micro-Autonomous  Systems  and  Technology  (MAST) 
CTA^,  through  DARPA  and  other  government  agencies,  and  through  small  contracts. 

ARL's  internal  research  focuses  on  human-machine  interaction,  the  development  of  air  and  ground  platforms,  increasing 
the  functionality  of  backpack-sized  ground  platforms  (e.g.,  increasing  their  ability  to  navigate  and  sense,  and  increasing 
their  capacity  for  distributed  networking  and  cognition),  and  developing  technologies  for  small-scale  mobility,  sensors, 
and  power.  The  Robotics  OTA  (ROTA)  is  concerned  primarily  with  increasing  the  capacity  for  autonomous  behavior  for 
large  platforms,  e.g.,  unmanned  vehicles  and  backpack-sized  platforms,  and  the  MAST  OTA,  with  developing  palm-sized 
autonomous  platforms.  An  important  issue  addressed  by  all  programs  is  enabling  cooperative  behavior  between  robots 
and  with  humans. 

The  ROTA  focuses  on  five  key  research  areas:  a  cognitively  based  world  model,  semantic  perception,  learning,  meta- 
cognition,  and  adaptive  behaviors.  Because  current  world  model  representations  are  relatively  shallow,  metrically  based, 
and  support  only  brittle  behaviors,  the  ROTA  is  creating  a  cognitive-to-metric  world  model  that  can  incorporate  and 
utilize  mission  context.  Current  perceptual  capabilities  for  unmanned  systems  are  generally  limited  to  a  small  number 
of  well-defined  objects  or  behaviors.  The  ROTA  is  raising  perception  to  a  semantic  level  that  enables  understanding  of 
relationships  among  objects  and  behaviors.  To  successfully  team  with  small  units,  the  command  and  control  of  unmanned 
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systems  must  move  away  from  the  current  hardware  controller  paradigm  to  one  of  verbal  and  gestural  communication, 
implicit  cues,  and  transparency  of  action  between  Soldier  and  robot.  The  RCTA  is  also  exploring  adaptive  behavior  and 
mechanics  that  permit  manipulation  of  arbitrarily  shaped  objects,  animal-like  mobility  in  complex  environments,  and 
conduct  of  military  missions  in  dynamic  tactical  conditions. 

As  stated  previously,  scale  imposes  fundamental  limits  on  system  design.  The  MAST  CTA  supports  and  conducts  research 
in  several  aforementioned  critical  areas.  These  include  microsystem  mechanics,  microelectronics,  signal  processing, 
power  and  energy,  and  the  integration  of  these  technologies  into  a  single  platform.  The  development  of  technologies 
for  power  and  energy,  in  particular  power  conditioning,  is  a  significant  internal  effort  at  ARL.  The  remaining  four  areas 
are  pursued  both  internally  and  externally  to  ARL,  and  jointly  with  ARL.  Issues  addressed  by  the  Alliance  include  how  to 
achieve  stable  aerodynamic  performance  in  unsteady  vortex-dominated  flows  at  low  Reynolds  numbers;  how  to  achieve 
animal-like  intelligence  and  navigation  with  limited  power,  limited  resolution  sensing,  limited  bandwidth,  and  low  level 
processing;  the  development  of  new  computing  architectures  to  insure  stable  and  reliable  processing  at  low  power;  the 
development  of  technologies  capable  of  managing  power  and  energy  sources  and  loads  on  small-scale  platforms;  and 
understanding  and  exploiting  intra-platform  interactions  and  efficiencies  in  a  collaborative  ensemble  of  small  robots. 

In  the  next  section,  we  considerthe  challenges  and  issues  in  increasingthe  capabilities  of  robotics  platforms  and  highlight 
the  contributions  made  by  ARL  researchers. 

4.  Capability  Challenges  and  Technological  Issues 

4.1.  Capability  Cliaiienges 

Careful  inspection  of  "The  Last  100  Meters"  reveals  numerous  fundamental  challenges  to  enabling  desired  autonomous 
capabilities.  For  example,  collaborative  behavior  across  mobile  air  and  ground  platforms  as  well  as  with  mounted  and 
dismounted  Soldiers  requires  a  stable,  mobile,  and  scalable  network.  Research@ARL  Network  Sciences  discusses  some 
of  ARL's  efforts  in  this  arena"^.  However,  network  infrastructure  and  protocols  must  account  for  the  limited  size  and  power 
of  small  platforms  in  comparison  to  ones  comparable  in  size  to  cars  or  small  planes,  which  can  port  conventional  network 
equipment. 

As  ground  platforms  transition  from  an  exterior  location  to  an  interior  one,  they  must  alter  their  gait  to  account  for  the 
change  in  ground  conditions  from  loosely  packed  soil  to  a  hard  surface.  If  the  structure  is  partially  collapsed,  they  must 
be  capable  of  climbing  over  or  avoiding  rubble.  Air  platforms  also  must  trim  their  control  surfaces  to  adjust  for  changes 
in  airflow  as  they  fly  through  windows. 

Platforms  that  enter  the  urban  structure  must  be  capable  of  altering  their  behavior  from  mobility  between  waypoints  to 
mobility  and  exploration  in  an  unmapped  area.  They  must  be  able  to  navigate  without  access  to  GPS  and  to  navigate 
and  explore  in  low-light  or  no-light  conditions.  Recognizing  these  capability  challenges  leads  to  the  list  of  technological 
challenges  addressed  in  the  next  section. 

4.2.  Technological  Issues 

In  this  section,  we  return  to  the  different  capabilities  that  characterize  a  robot  and  provide  more  discussion. 
4.2.1.  Mobility 

Without  the  ability  to  move,  the  platforms  we  have  been  discussing  are  simply  highly  capable  sensing,  computing,  and 
communication  platforms.  The  Army's  primary  interests  are  developing  terrestrial  and  aerial  platforms,  and  not  aquatic 
ones.  Terrestrial  platforms  may  use  wheels,  tracks,  or  legs  to  provide  forward  motion.  The  choice  of  mechanism  depends 
upon  the  size  of  the  platform  and  its  application.  It  is  also  possible  for  a  platform  not  to  have  any  wheels,  track,  or  legs 
at  all  and  to  mimic  the  body  undulation  of  a  snake,  which  provides  mechanism  for  both  lateral  motion  and  climbing. 
Other  means  to  achieve  vertical  motion  include  jumping  and  hopping.  For  multi-legged  platforms  that  mimic  the  jumping 
behavior  of  lizards,  tails  are  also  an  important  appendage.  Aerial  platforms  may  use  fixed,  flapping,  or  rotary  wings  to  fly 
or  hover,  and  designers  may  combine  modalities,  e.g.,  legs  and  wings,  to  produce  hybrid  platforms. 

One  of  the  primary  issues  associated  with  mobility  is  scale.  Large,  vehicle-sized  autonomous  platforms  are  simply 
unmanned  vehicles  and  most  backpack-sized  platforms  are  scaled-down  versions  of  large  vehicles.  This  scaling  is 
possible  due  to  the  development  of  small,  efficient  motors. 
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For  palm-sized  platforms,  designers  look  to  biology  for  solutions;  physics  and  manufacturing  make  it  difficult  to  scale 
down  existing  solutions  further.  Thankfully,  nature  provides  a  multitude  of  choices.  For  example,  avian  wing  flapping 
differs  considerably  from  that  of  insect  wing  flapping.  Only  at  scales  that  are  comparable,  such  as  the  humming  bird  and 
the  hawk  moth,  is  the  flapping  motion  similar. 

Further,  due  to  the  size  of  individual  components,  traditional  manufacturing  and  assembly  methods  are  not  possible. 
Even  if  they  were,  since  the  ratio  of  surface  area  to  volume  increases  as  spatial  scale  decreases,  friction  increases  and 
makes  solutions  for  large-scale  platforms  inefficient  at  small  scales.  Thus,  new  designs  and  new  fabrication  techniques 
are  necessary. 

To  enable  small-scale  mobility,  ARL  focuses  primarily  on  MEMS-based  technologies  to  actuate  walking,  flying,  and  jumping, 
and  for  navigation  control  of  flight.  ARL's  expertise  in  thin  film  lead  zirconate  titanate  (PZT),  a  piezoelectric  material 
capable  of  generating  an  electric  potential  when  a  mechanical  force  is  applied  and,  conversely,  generating  a  mechanical 
strain  when  an  electric  field  is  applied,  is  extensively  used  for  most  mechanisms.  Jumping  mechanisms  additionally  use 
an  energetic  silicon-based  actuator  to  generate  vertical  thrust.  Together,  this  research  enables  mobility  at  the  millimeter 
scale. 

To  mimic  biology,  actuators  for  small-scale  mobility  must  achieve  large  force  and  displacement  at  low  voltages  and 
consume  little  power,  a  key  feature  of  PZT.  In  addition,  one  can  fabricate  PZT-based  actuators  in  series  or  parallel 
to  generate  large  forces  or  wide  angular  ranges  of  motion.  This  is  critical  to  realize  complex  mechanisms  for  lateral, 
rotational,  and  vertical  movement. 

The  review  article  "PZT-Based  Piezoelectric  MEMS  Technology,"  written  by  several  members  of  ARL's  MEMS  team  with 
lead  author  Gabriel  Smith  (page  13),  provides  an  overview  of  PZT,  the  fabrication  of  micro-devices  from  PZT,  and  their 
application  to  enabling  small-scale  mobility,  in  addition  to  other  applications  such  as  mechanical  logic  and  small  motors. 
The  paper  discusses  actuator  design  for  multipedal  locomotion,  winged-insect  flight,  and  an  artificial  haltere  for  measuring 
the  rate  of  angular  rotation  on  winged  platforms. 

Details  of  the  actuator  design  for  multipedal  locomotion  are  discussed  in  the  joint  authored  papers  "Modeling  and  Optimal 
Low-Power  On-Off  Control  of  Thin-Film  Piezoelectric  Rotational  Actuators,"  with  Biju  Edamana  as  lead  author  (page  31), 
and  "Multi-Degree-of-Freedom  Thin-Film  PZT-Actuated  Microrobotic  Leg,"  with  Choong-Ho  Rhee  as  lead  author  (page  45). 

To  lift  legs,  the  authors  use  traditional  vertical  unimorph  actuators  and  out-of-plane  rotational  and  torsional  actuators. 
For  rotational  joints,  the  authors  use  lateral  piezoelectric  actuators.  Due  to  inherent  nonlinearities  in  PZT,  the  structures 
are  optimal  only  for  small  displacements.  Thus,  flexural  amplification  is  required  to  translate  large  forces  into  large 
displacements.  The  individual  flexural  actuators,  piezoelectric  actuators,  and  coupled  flexures,  can  provide  only  -5°  of 
rotation  and  are  therefore  cascaded  to  create  joints  capable  of  larger  rotations.  Lithography  allows  one  to  combine  thin 
film  actuators  monolithically  to  produce  high  performance  flexural  and  structural  elements. 

In  Smith  etal.  (page  13),  ARL  researchers  discuss  the  design  and  fabrication  of  2-mm  long  PZT-based  wings  that  reproduce 
the  aerodynamic  forces  generated  by  agile  flying  insects.  The  scale  of  the  wings  is  comparable  to  that  of  a  fruit  fly.  The 
system  uses  thin-film  PZT  actuators  with  two  independent  degrees-of-freedom  to  drive  a  micro-fabricated  wing  structure. 
A  stroke  actuator  drives  the  flapping  mechanism  at  resonance  and  the  pitch  actuator  quasi-statically  drives  wing  rotation. 
The  coordinated  motion  of  these  two  and  the  resulting  three-dimensional  wing  kinematics  generate  the  aerodynamic  lift 
that  is  capable  of  sustaining  flight.  ARL  has  demonstrated  120°  of  stroke  amplitude  using  a  10  V  bias  and  45°  of  pitch 
amplitude  using  a  20  V  bias  at  frequencies  comparable  to  those  in  nature. 

The  ARL  team  also  describes  in  Smith  et  al.  (page  13)  the  construction  of  an  artificial  haltere,  a  sensor  that  measures 
body  rotation  in  some  two-winged  insects.  The  haltere  consists  of  a  pendulum-like  structure  that  oscillates  in  a  direction 
orthogonal  to  forward  motion.  When  the  insect  turns,  the  resistance  of  the  pendulum  to  the  change  in  the  direction  of 
motion  generates  a  force  measured  by  hairs  at  the  base  of  the  haltere.  The  hairs,  which  are  connected  to  the  insect's 
neural  system,  provide  the  insect  with  an  inertial  measurement  of  its  position  in  space.  Inspired  by  the  insect  sensors. 
Smith  et  al.  fabricated  a  copper  pendulum  that  they  actuated  using  PZT.  The  PZT  actuator  also  measured  the  forces 
produced  by  changes  in  angular  position.  Simulation  and  component  testing  indicate  the  artificial  haltere  is  sensitive  to 
changes  in  angular  rate  on  the  order  of  60  deg/s.  Smith  et  al.  also  fabricated  halteres  in  coupled  arrays  to  enable  triaxial 
measurements. 


8 


In  "The  First  Launch  of  an  Autonomous  Thrust-Driven  Microrobot  Using  Nanoporous  Energetic  Silicon,"  with  lead  author 
Wayne  Churaman  (page  59),  ARL  researchers  describe  the  fabrication  of  a  300-mg,  4  mm  x  7  mm  x  0.5  mm  robotic 
system  that  reaches  a  vertical  height  of  80  mm  using  about  0.5  |jl  of  nanoporous  energetic  silicon.  Energetic  silicon, 
which  stores  energy  chemically,  is  an  alternative  to  storing  mechanical  energy  in  a  spring.  This  reduces  system  complexity 
by  eliminating  the  need  for  an  additional  actuator  to  compress  a  spring.  The  platform  uses  a  light-sensitive  detector  to 
initiate  the  reaction  that  converts  chemical  energy  into  mechanical.  Monolithic  design  and  fabrication  enables  a  device 
with  integrated  control,  sensing,  power,  and  actuation,  which  is  critical  for  large-scale  production  of  small-scale  robotic 
platforms. 

4.2.2.  Navigation  and  Controi 

For  all  platforms,  developing  the  capacity  for  controlled  movement  without  human  intervention  is  a  primary  issue.  The 
Wright  Brothers  receive  credit  for  inventing  flight  primarily  because  they  developed  mechanisms  for  making  a  controlled 
turn.  Presently,  due  to  our  incomplete  understanding  of  the  physics,  small-scale  platforms  lack  well-developed  controls 
for  mobility.  Small-scale  aeromechanics  and  avionics  are  necessary  to  control  small  platforms  in  wind  and  in  gusts,  and 
as  they  transition  from  a  structure's  exterior  to  its  interior.  Similarly,  as  small  legged-terrestrial  platforms  encounter 
different  ground  conditions,  they  must  alter  their  gait,  which  requires  advances  in  the  new  science  of  terra-mechanics 
and  in  control  systems  based  on  this  new  science.  Once  capable  of  controlled  movement,  the  platforms  must  navigate 
on  their  own. 

A  fundamentally  important  problem  in  control  systems  is  handling  uncertainty,  unexpected  external  disturbances,  and 
system  model  uncertainties.  Examples  of  disturbances  include  wind  gusts  that  strongly  perturb  small  aerial  vehicles,  and 
wheel  or  track  slippage  in  rough  and  uneven  terrain.  George,  Singia,  and  Crassidis  address  the  problem  of  robust  control 
(page  69)  by  expanding  on  a  Kalman  filter  based  adaptive  disturbance  accommodating  stochastic  control  scheme.  Their 
approach,  under  minimal  assumptions  on  the  disturbance,  simultaneously  estimates  both  the  system  states  and  the 
model-error  encountered.  Here  the  model-error  dynamics  are  unknown,  so  a  second  Kalman  filter  is  incorporated  into 
the  control  scheme  to  adaptively  estimate  the  disturbance  statistics.  This  method  is  shown  by  the  authors  to  be  provably 
asymptotically  optimal  for  linear  systems  with  bounded  external  disturbance,  and  is  readily  implemented  as  a  supplement 
to  a  conventional  Kalman  filter  controller. 

The  stochastic  controller  idea  was  generalized  by  George  (page  97)  to  the  case  of  a  Kalman  filter  operating  in  continuous 
time  while  under  a  persistent  excitation.  The  author  develops  a  control  method  that  is  provably  convergent  in  the  sense  of 
estimating  the  unknown  error  statistics,  even  when  the  input  excitation  is  unknown,  and  thus  the  method  asymptotically 
recovers  the  desired  optimal  performance. 

Many  applications  of  autonomy  are  significantly  complicated  by  the  need  for  wireless  networking.  Mobile  ad  hoc  networks 
without  commercial  infrastructure  face  major  challenges  in  scalability  and  robustness.  Radio  accomplishes  this  but 
suffers  from  signal  strength  fluctuation  with  small  movement  on  the  order  of  a  wavelength  of  the  center  frequency,  as  well 
as  attenuation  that  is  generally  well  beyond  free-space  loss.  ARL  scientists  have  addressed  this  problem  by  developing 
control  strategies  to  measure,  map,  and  maintain  wireless  connectivity  while  carrying  out  autonomous  missions. 

ARL  researchers  Twigg,  Fink,  Yu,  and  Sadler  (page  105)  have  developed  and  tested  sensing  and  control  algorithms 
to  autonomously  discover  the  radio  connectivity  region  of  a  radio  base  station  without  any  a  priori  knowledge  of  the 
environment.  The  authors  demonstrate  a  method  for  simultaneously  exploring,  developing  a  physical  map,  and  overlaying 
a  connectivity  map,  such  that  within  the  connectivity  area  communications  with  the  base  station  can  be  accomplished 
with  a  certain  quality  guarantee.  The  method  exploits  the  observed  radio  signal  strength  as  it  varies  with  motion,  and 
the  authors  develop  an  efficient  method  that  rapidly  finds  and  explores  the  connectivity  boundary  contour  using  the 
radio  signal  strength  gradient.  The  resulting  control  algorithms  are  a  significant  first  step  towards  larger  problems  of 
connectivity  control  during  autonomous  operations  in  unknown  environments,  including  the  networking  of  autonomous 
agents  and  Soldiers. 

4.2.3.  Human-Mactiine  Coiiaboration 

The  use  of  multiple  autonomous  platforms  is  inspired  by  bionetworks  in  nature  such  as  insect  swarms.  Although  each 
single  entity  in  a  swarm  has  limited  intelligence,  the  swarm  appears  to  behave  with  intent  when  the  entities  function  in 
concert.  Fish  and  birds  are  examples  of  animals  whose  collective  behavior  belies  their  limited  intelligence. 
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Although  the  swarm  analogy  is  intellectually  appealing,  differences  exist  between  bionetworks  and  networks  of  autonomous 
mobile  platforms.  Put  simply,  biological  swarms  communicate  simple  messages  primitively  to  perform  primitive  behaviors. 
Whereas  some  tasks  performed  by  robots  can  be  addressed  using  simple  bottom-up  bio-mimetic  control,  others  that 
require  complex  messaging  to  achieve  complex  behaviors  might  be  better  suited  to  top-down  centralized  control  in  which 
a  single  intelligent  entity  controls  a  layered  hierarchical  network  wherein  each  layer  contains  more  entities  of  lesser 
intelligence  than  the  layer  above  it.  From  a  research  perspective,  we  need  to  explore  a  range  of  possibilities  and  designs. 

Today  the  joint  human-robot  role  includes  load-carrying,  casualty  extraction,  hazardous  material  detection,  counter 
explosive  devices,  building  mapping/clearing  and  firefighting.  Tomorrow,  Soldiers  may  be  required  to  navigate  a  robot 
while  concurrently  conducting  surveillance,  maintaining  local  security  and  situation  awareness,  and  communicating  with 
fellow  team  members.  The  ARL  scientists  Chen  and  Barnes  collaborated  with  Haper-Sciarini  on  the  critique  "Supervisory 
Control  of  Multiple  Robots:  Human-Performance  Issues  and  User-Interface  Design"  (page  119).  This  publication  reviews 
research  done  on  the  limitations  and  advantages  of  supervisory  control  of  unmanned  air  and  ground  vehicles. 

Humans  exert  supervisory  control  of  technology  through  the  level  of  autonomy  allowed  and  through  interactions  with 
the  automated  technology  that  includes  planning,  teaching,  monitoring,  intervening  and  learning.  The  Chen  et  al. 
paper  examines  human  performance  issues  in  supervisory  control  of  robotic  vehicles  including  things  such  as  operator 
multitasking  performance,  trust  in  automation,  situational  awareness  and  operator  workload,  and  reviews  user-interface 
solutions  that  could  potentially  address  these  issues.  The  importance  of  understanding  the  interface  between  humans 
and  robots  in  order  to  determine  what  we  can  and  cannot  control  is  emphasized,  anticipating  the  increasing  number  of 
tasks  that  humans  cannot  perform  alone. 

Conducting  large-scale  experiments  involving  multiple  people  interacting  with  robots  over  various  terrains  is  costly  and 
the  resulting  vast  amount  of  data  is  difficult  to  process  and  interpret.  Consequently,  before  conducting  scenarios  in 
which  robot  tasks  have  increasing  complexity  it  is  advisable  to  guide  their  design  through  computer  simulations  with 
human  interactions.  Chen  and  Barnes  of  ARL  tested  some  ideas  concerning  the  management  of  ground  robots  with  the 
assistance  of  an  intelligent  agent  capable  of  coordinating  the  robots  and  changing  their  routes  on  the  basis  of  battlefield 
developments  in  "Supervisory  Control  of  Multiple  Robots:  Effects  of  Imperfect  Automation  and  Individual  Differences" 
(page  139).  They  did  this  using  a  military  multitasking  environment  simulation  of  an  intelligent  agent,  RoboLeader, 
assisting  the  performance  of  supervising  multiple  robots  to  complete  military  reconnaissance  missions  in  a  dual-task 
and  multitasking  environment. 

Chen  and  Barnes  were  particularly  concerned  with  the  influence  of  agent  imperfections,  such  as  rates  of  false  alarm  and 
the  propensity  of  the  agent  to  miss  the  target,  on  the  operator's  performance  of  tasks,  for  example,  on  target  detection, 
route  editing  and  situation  awareness.  They  found  that  participants'  attentional  control  and  video  gaming  experience 
significantly  affected  their  overall  multitasking  performance.  They  determined  that  intelligent  agents,  such  as  RoboLeader, 
can  benefit  the  overall  human-robot  teaming  performance  when  perfectly  reliable.  On  the  other  hand,  the  effects  of  agent 
imperfections  can  be  significant  and  difficult  to  anticipate  and  further  research  needs  to  be  done  in  this  area. 

An  important  factor  in  determining  the  utility  of  intelligent  agents  in  supporting  the  dynamic  robot  re-tasking  based 
on  battlefield  development  is  the  level  of  assistance  provided  by  the  agent  to  the  robotics  operator.  In  "Supervisory 
Control  of  Multiple  Robots  in  Dynamic  Tasking  Environments"  (page  159),  Chen  and  Barnes  of  ARL  systematically  change 
the  level  of  assistance  provided  by  RoboLeader  and  find  that  participants'  primary  task  benefitted  from  all  levels  of 
assistance  compared  to  manual  performance.  Here  again  frequent  video  gamers  demonstrated  significantly  better 
situation  awareness  of  the  mission  environment  than  did  infrequent  gamers.  Moreover,  the  participants  experienced 
lower  workload  when  they  were  assisted  by  RoboLeader  than  when  they  performed  the  target  entrapment  task  manually. 
This  study  increased  the  understanding  of  the  interplay  among  the  level  of  autonomy,  multitasking  performance  and 
individual  differences  in  military  tasking  environments. 

The  results  suggest  that  agent/human  supervision  of  multiple  robots  permits  synergy  without  usurping  the  human's 
decision-making  authority  and  those  levels  of  assistance  with  less  than  full  autonomy  can  be  as  effective  as  fully 
autonomous  levels  of  assistance.  Stated  differently  it  enables  the  person  to  exceed  the  classical  (seven  ±  two)  finding  for 
span  of  apprehension  that  would  traditionally  overwhelm  cognitive  resources  and  with  the  assistance  of  the  intelligent 
agent  successfully  expand  their  workload. 

For  a  team  of  humans  and  robots  to  complete  a  mission  effectively,  humans  must  trust  that  a  robotic  teammate  will 
protect  their  interests  and  welfare.  Trust  directly  affects  the  willingness  of  people  to  accept  robot-produced  information 
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and  follow  robot  suggestions.  This  affects  the  decisions  humans  make  in  uncertain  and  risky  environments.  The  less 
an  individual  trusts  a  robot,  the  sooner  he  or  she  will  intervene  as  the  robot  progresses  toward  task  completion.  Most 
current  studies  on  trust  attempt  to  measure  and  quantify  the  level  of  trust  that  exists  for  a  particular  platform  and 
human-robot  team  configuration.  Hancock,  Billings  and  Schaefer  of  the  University  of  Central  Florida,  together  with  Chen 
of  ARL  and  Visser  and  Parasuraman  of  George  Mason  University  evaluate  and  quantify  the  effects  of  human,  robot,  and 
environmental  factors  on  perceived  trust  in  human-robot  interaction  in  "A  Meta-Analysis  of  Factors  Affecting  Trust  in 
Human-Robot  Interaction"  (page  177). 

Hancock  et  al.  replace  a  historically  qualitative  and  descriptive  review  of  trust  with  one  that  is  quantitative.  They  conclude 
that  factors  related  to  the  robot  performance  had  the  greatest  current  association  with  trust  while  those  related  to  the 
environment  were  only  moderately  associated  with  trust.  Moreover,  there  was  little  evidence  that  trust  was  dependent  on 
the  human-related  factors.  The  study  provides  an  empirical  foundation  on  which  to  advance  both  theory  and  application. 

5.  Other  Issues 

5.1.  Sensors  and  Signal  Processing 

ARL,  along  with  its  partners,  performs  considerable  work  in  sensors  and  signal  processors  for  robotics  and  herein  we 
highlight  some  of  the  important  aspects  that  are  being  addressed. 

With  regard  to  sensors,  it  is  important  to  distinguish  between  sensors  for  platform  control,  for  example,  using  a  mechanical 
sensor  like  a  haltere  to  provide  stable  flight,  versus  sensors  to  complete,  for  example,  a  surveillance  mission.  Vision 
sensors,  in  particular,  can  be  used  either  for  navigation,  i.e.,  to  detect  and  avoid  obstacles,  or  they  can  be  used  for 
situation  awareness,  to  classify,  recognize,  or  identify  objects.  For  low-light  or  no-light  navigation  or  situational  awareness, 
radar  is  a  potential  solution  so  long  as  it  can  be  made  small  enough  to  fit  on  a  platform.  The  same  is  true  of  hair-like 
mechanical  sensors.  If  they  can  be  made  small  enough,  artificial  hairs  are  useful  either  for  navigation  or  for  detecting  or 
measuring  wind. 

An  overriding  issue  with  regard  to  the  development  of  sensors  is  whether  they  can  provide  the  requisite  accuracy  and 
precision  subject  to  the  constraints  of  size,  weight,  and  power.  The  accuracy  and  precision  required  depends  upon  the 
number  of  platforms  working  collaboratively.  Even  if  a  single  sensor  has  poor  resolution,  by  combining  measurements 
from  multiple  platforms,  one  can  improve  the  effective  resolution. 

If  one  needs  a  robot  to  alter  and  manipulate  its  environment,  in  addition  to  sensors,  the  robot  requires  a  mechanism  to 
do  so.  For  example,  the  Mars  rovers  alter  their  environment  using  drills  and  scoops  to  access  material  below  the  planet's 
surface  but  they  also  use  lasers  to  ablate  material  on  its  surface.  Thus  gripping  and  moving  an  object  require  tactile 
sensors  and  single  or  multiple  arm-like  appendages,  multiple  finger-like  appendages,  or  both.  Articulation,  actuation, 
and  control  of  these  appendages  are  critical  issues,  as  is  selecting  materials  that  are  strong,  compliant,  and  durable  for 
repeated  use. 

Sensors  by  themselves  provide  only  measurements  pertaining  to  the  environment.  For  autonomous  behavior,  it  is 
necessary  to  process  these  physical  measurements  to  give  them  meaning.  Thus,  specialized  dedicated  processors,  for 
example,  for  vision,  mapping,  planning,  and  navigation  are  also  required.  There  exists  also  a  strong  link  between  the 
requirement  for  integrated,  low-power  but  high  performance  sensors  and  the  photonics  and  electronics  necessary  to 
enable  them. 

Sensors  impacts  communications  as  well.  What  is  transmitted,  measurements,  processed  data,  or  commands,  depends 
upon  the  processing  and  communications  power  available  to  a  single  platform.  Simple  messages  require  complex 
processing,  whereas  measurements  require  large  bandwidths.  Researchers  are  considering  whether  it  is  more  efficient 
to  transmit  data  to  a  single,  high  performance  processor,  or  provide  each  platform  with  a  moderate  amount  of  processing 
to  reduce  the  bandwidth  and  energy  required  for  transmission.  Latency  is  also  an  issue  designers  must  consider  in  this 
trade-off.  The  amount  of  latency  incurred  and  the  amount  of  latency  that  can  be  tolerated  before  the  distributed  control 
of  the  collective  becomes  unstable  is  a  critical  research  issue. 

5.2.  Non-technicai  Issues 

ARL's  purview  is  to  solve  the  technical  hurdles  limiting  the  performance  of  autonomous  platforms.  However,  there  exist 
nontechnical  issues  that  also  limit  the  acceptance  of  robots  in  military  operations.  The  obvious  and  most  concerning 
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is  their  use  as  a  remote  weapon  system.  Government  lawyers  and  policy  makers,  along  with  academicians,  are  already 
studying  the  legal  and  ethical  issues  associated  with  this. 

As  indicated  in  Hancock  et  al.,  trust  is  another  significant  nontechnical  issue.  However,  trust  can  be  expected  to  be  gained 
over  time  as  Soldiers  have  more  interaction  with  robots  and  learn  what  they  can  and  cannot  do  well,  and  when  and  how 
they  should  be  used.  Trust  will  also  increase  as  scientists  and  engineers  gain  understanding  and  improve  designs  and 
reliability. 

Army  planners  must  consider  the  advantages  and  disadvantages  of  robotics,  particularly  as  the  technology  changes  over 
time.  The  increase  in  effectiveness  must  outweigh  any  changes  in  mission  structure  and  logistics  to  support  the  inclusion 
of  robotics.  Is  it  more  efficient  and  effective  to  use  a  robotic  platform  as  opposed  to  other  more  conventional  means?  For 
example,  is  it  more  effective  to  sustain  a  robotic  mule  in  the  field  or  a  real  one?  Ultimately,  logisticians  need  to  determine 
whether  the  gross  weight  of  a  Brigade  Combat  Team  increases  or  decreases  when  it  uses  robotic  platforms  to  provide 
new  capabilities  or  enhance  old  ones,  and  planners  need  to  assess  whether  the  improvement  in  performance  warrants 
an  increase  in  weight. 

Finally,  as  ARL  seeks  to  integrate  robotic  platforms  into  teams  of  soldiers,  others  seek  to  integrate  robotics  even  more 
intimately  with  biology.  In  DARPA's  HI-MEMS  program,  for  example,  researchers  implanted  electronics  into  the  larvae  of 
a  moth  that  the  researchers  used  to  control  the  moth's  flight  as  an  adult. 

We  are  pleased  to  share  the  following  collection  of  articles  that  highlight  the  advances  ARL's  researchers  have  made  in 
the  pursuit  of  enabling  truly  autonomous  systems. 
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This  review  article  presents  recent  advancements  in  the  design 
and  fabrication  of  thin-film  (<3  jim)  lead  zirconate  titanate 
(PZT)  microelectromechanical  system  (MEMS)  devices.  The 
article  covers  techniques  for  optimizing  highly  (001)/(100)  ori- 
ented chemical  solution  deposited  PZT  films  to  achieve 
improved  piezoelectric  coefficients.  These  PZT  films  combined 
with  surface  and  bulk  micromachining  techniques  are  fabri- 
cated into  actuators  and  transducers  for  radio  frequency  (RF) 
switches,  nanomechanical  logic,  resonators,  and  power  trans- 
formers for  use  in  communication  systems  and  phased-array 
radar.  In  addition,  the  large  relative  displacements  generated 
by  PZT  thin  films  have  been  used  to  demonstrate  mechanical 
mobility  in  MEMS  devices,  including  insect-inspired  flight 
actuators  and  ultrasonic  travefing  wave  motors.  In  conjunction 
with  actuation,  PZT  films  are  being  developed  for  feedback 
sensors  for  the  integrated  control  of  insect-inspired  robots. 

I.  Introduction 

THE  versatility  of  thin-film  lead  zirconate  titanate, 
PbZr;tTii_^03  or  PZT,  as  a  material  in  microelectrome- 
chanical system  (MEMS)  devices  has  expanded  greatly  in  the 
past  several  years.  ^  PZT  thin  films,  deposited  by  both  sput- 
tering and  sol-gel,  have  been  researched  for  MEMS  as  well 
as  other  electronic  device  technologies  for  nearly  30  years. 
As  a  result,  nonvolatile  ferroelectric  random  access  memory 
(FRAM)  is  now  commercially  available  from  several  vendors 
including  Ramtron,  Texas  Instruments,  and  Fujitsu. 
Although  not  a  MEMS  technology,  FRAM  demonstrates 
successful  integration  of  high  performance  ferroelectric  thin 
films  with  complementary  metal-oxide  semiconductor 
(CMOS)  integrated  circuit  technology.  FRAM  devices  are 
built  by  adding  the  metal-ferroelectric-metal  stack  on  top  of 
the  CMOS  circuitry  in  a  two  photomask  process.  This  addi- 
tion to  CMOS  creates  a  highly  useful  integrated  product  for 
embedded  memory  appHcations  and  provides  a  soHd  back- 
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ground  and  experience  on  which  to  base  the  design  and  fab- 
rication of  PZT-based  MEMS  devices. 

Two  PZT  thin-film  based  MEMS  technologies  are  starting 
to  make  their  way  into  the  commercial  market.  Panasonic 
(Kadoma,  Osaka,  Japan)  recently  developed  the  EWTS9T  ser- 
ies thin-film  PZT  16-mm^  two-axis  gyroscope  targeted  for 
consumer  electronics  applications.^  In  addition,  several  com- 
panies, including  Epson  (Suwa,  Nagano,  Japan)  and  FUJI- 
FILM  Dimatix  (Santa  Clara,  CA)  are  offering  inkjet  print 
heads  using  thin  film  PZT.^'^^  The  state  of  advanced  product 
development  of  both  of  these  devices  indicates  the  market  via- 
bility and  advantages  of  PZT-based  MEMS  devices  over  com- 
peting technologies. 

This  article  is  intended  to  provide  an  overview,  by  no 
means  all  inclusive,  of  the  types  of  new  devices  that  are 
enabled  by  PZT-based  MEMS  with  specific  attention  to 
devices  targeting  radio  frequency  (RF)  and  small  scale  robot- 
ics systems.  Although  most  of  the  fabrication  processes  are 
common  across  the  devices,  the  appHcation  areas  are  extre- 
mely diverse.  The  first  section  will  cover  fabrication  and 
material  processing  topics.  The  second  section  of  the  article 
will  focus  on  actuation  and  sensing  for  the  use  in  millimeter- 
scale  robotics.  The  final  section  of  the  article  will  outUne 
appHcation  areas  in  radio  frequency  microelectromechanical 
systems  (RF  MEMS)  devices. 

II.    Fabrication  and  Material  Optimization 

(1)    Fabrication  Overview 

Before  discussing  specifics  for  any  particular  material,  a  gen- 
eric piezoelectric  MEMS  (PiezoMEMS)  fabrication  process 
will  be  presented.  In  this  general  process,  all  materials  com- 
prising the  piezoelectric  composite  are  deposited  as  uniform 
blanket  coatings  onto  the  substrate  of  choice  (see  Figure  1). 
First,  a  substrate  must  be  chosen  specific  to  the  targeted  final 
device.  The  most  common  substrates  are  (100)  silicon  sub- 
strates. For  RF  applications,  the  resistivity  of  the  substrate 
must  be  >  5000  or  even  as  high  as  10  000  Ohm-cm  depending 
on  the  frequency  of  interest.  Another  option  is  sihcon-on- 
insulator  (SOI)  substrates  that  are  extremely  attractive  for 
resonator/filter  and  robotics  applications.  For  the  SOI  sub- 
strates, the  device  layer  silicon  can  vary  between  2  and 
>  40  )im  depending  on  the  application  requirements.  Other 
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Fig.  1.  Schematic  cross-sections  of  unimorph  piezoelectric  composite 
starting  wafers  on  both  (a)  Si  and  (b)  siHcon-on-insulator  (SOI) 
substrates. 


substrate  choices  are  possible,  but  sihcon  and  SOI  will 
remain  the  focus  of  this  article. 

Following  the  substrate  choice,  an  appropriate  insulating 
layer  is  deposited  to  serve  as  a  protective  barrier  between  the 
subsequent  metal  electrode  and  silicon  substrate.  In  addition, 
this  layer  can  serve  as  a  mechanical  elastic  layer  for  a  unim- 
orph piezoelectric  actuator.  If  used  as  the  elastic  layer, 
proper  design  of  the  elastic  layer  composite  will  result  in  con- 
trolled residual  stress  induced  deformation  in  the  final  device 
configuration. ^^'^^  As  an  example,  see  Fig.  2,  in  which  the 
location  of  a  highly  stressed  sihcon  nitride  thin  film  is  used 
to  modify  the  postfabrication  static  deformation  in  a  Si02/ 
Si3N4/Si02/Ti/Pt/PZT/Pt  piezoelectric  composite.  Placing  the 
tensile  silicon  nitride  further  from  the  neutral  axis  of  the 
structure  preferentially  results  in  curvature  toward  the  nitride 
layer  in  the  final  device  and  can  be  utilized  to  ensure  proper 
device  operation. 

The  remaining  steps  for  the  piezoelectric  composite  are 
associated  with  the  actuator  and  electrode  materials.  For  a 
parallel  plate  configuration,  where  the  PZT  is  sandwiched 
between  two  electrodes,  the  most  common  choice  for  the  bot- 
tom electrode  layer  is  sputtered  Pt  deposited  on  top  of  a  buffer 
layer  of  titanium  dioxide.  Alternatively,  conductive  oxide  elec- 
trodes, either  IrO^^^  and  RuO;^/'^  can  be  utilized.  These  mate- 
rials are  common  for  FRAM  where  ferroelectric  fatigue  has  a 
significant  role  in  overall  device  performance  and  rehability. 
The  PZT  layer  can  be  deposited  by  a  variety  of  methods 
including  sputtering,  metal-organic  chemical  vapor  deposition, 
and  chemical  solution  deposition  with  thicknesses  ranging 
from  0.5  to  2.0  \im  depending  on  device  design  parameters. 
Finally,  the  top  electrode  is  deposited,  preferentially,  by  sput- 
tering. Similar  to  the  bottom  electrode,  Pt,  IrOx,  and  RuOx 
are  the  most  common  choices.  Cr/Au  is  also  utilized,  especially 
for  experimentation  or  device  evaluation  purposes.  Note,  for 
improved  ferroelectric  fatigue  (for  FRAM  and  any  apphcation 
requiring  bipolar  operation  of  the  PZT)  only  one  of  the  elec- 
trodes is  required  to  be  a  conductive  oxide. 

The  following  description  of  a  generic  fabrication 
process  flow  is  similar  to  that  for  devices  fabricated  at  the 
Specialty  Electronic  Materials  and  Sensors  Cleanroom  Facil- 
ity, U.S.  Army  Research  Laboratory  (ARL),  Adelphi,  MD. 
The  general  fabrication  process  (see  schematic  in  Fig.  3)  for 


PiezoMEMS  devices  at  ARL  uses  the  following  process  flow 
with  minor  variations  (i.e.,  release  process,  layer  thickness, 
and  process  sequence)  to  meet  unique  requirements  of  a  par- 
ticular device.  For  more  information,  specific  to  a  particular 
device,  see  references  discussed  in  the  representative  device 
section.  The  actuator  is  commonly  patterned  using  four  indi- 
vidual process  steps.  The  first  step  is  to  pattern  the  top  plati- 
num electrode  with  the  argon  ion-milhng  in  a  4Wave  Ion 
Mill.  This  etch  is  then  followed  by  an  additional  ion-milling 
of  the  PZT  and  bottom  electrode  features.  To  open  access  to 
the  bottom  electrode,  a  via  is  patterned  by  ion-milhng  using 
an  endpoint  etch  to  stop  on  the  platinum  layer.  After  ion- 
milling,  a  quick  wet  etch  is  used  to  clean  up  any  remaining 
PZT  related  compounds  on  the  bottom  platinum  electrode. 
The  actuator  or  sensor  structure  is  then  further  defined  by 
patterning  the  composite  elastic  layer  (either  the  dielectrics  or 
dielectrics  plus  device  layer  Si)  with  a  reactive  ion  etch  to 
provide  access  to  the  sihcon  substrate  for  the  eventual  release 
etch.  Next,  Metal  2  is  deposited  and  patterned.  Metal  2  con- 
sists of  a  bi-layer  of  titanium  and  gold  or  chromium  and 
gold  deposited  with  electron  beam  evaporation  and  patterned 
via  hftofif.  Metal  2  commonly  defines  the  co-planar  wave- 
guide transmission  fines  for  RF  devices,  contact  structures, 
and  anchor  features  required  for  gold  air  bridges.  In  this  pro- 
cess flow.  Metal  3  is  used  to  create  air-bridge  and  cross-over 
structures.  To  create  these  structures,  a  photo-resist  sacrificial 
layer  is  patterned  and  cured  followed  by  the  deposition  and 
Hft-ofif  of  2  jam  gold  thin  film  to  form  Metal  3.  If  required, 
the  device  layer  silicon  or  the  handle  wafer  on  sihcon-on- 
insulator  wafers  are  patterned  and  etched  via  silicon  deep 
reactive  ion  etching.  In  the  final  process  step,  a  XeF2  etch  is 
used  to  either  etch  the  device  layer  or  regions  of  the  handle 
wafer  to  create  a  released  PiezoMEMS  device.  It  should  be 
noted  that  in  many  instances,  an  atomic  layer  deposited  alu- 
mina thin  film  is  used  to  provide  protection  to  the  exposed 
PZT  areas  as  well  as  provide  XeF2  etch  protection  of  the 
device  Si  areas. After  processing,  the  devices  are  typically 
poled  with  a  10-20  V  DC  or  1  Hz  AC  field  for  a  period  of  3 
-5  min.  In  typical  0.5  jam  PZT,  reverse  coercive  fields  can  be 
introduced  by  as  little  as  -2  V  so  0-15  V  unipolar  positive 
voltage  AC  operation  is  the  common  actuation  input.  For 
most  devices,  at  0.5-2)im  thickness,  unipolar  operation  is 
above  the  coercive  field  and  the  PZT  poles  to  an  exponential 
Hmit  over  time. 


(2)    Elastic  Layer  and  Pt  Metallization 

As  described  in  the  previous  process  flow,  the  elastic  dielec- 
tric layer  and  bottom  platinum  electrode  are  the  first  layers 
deposited  on  the  substrate  and  have  proven  to  have  signifi- 
cant impact  on  the  properties  of  the  subsequent  PZT  film. 
The  use  of  epitaxial  templates  is  critical  to  define  well  ori- 
ented PZT  for  improved  piezoelectric  coefficients.^^  The  syn- 
thesis of  Ti02/Pt  bottom  electrodes  on  a  well  defined  elastic 
layer  of  thermal  Si02  grown  on  Si  (100)  serves  as  a  template 


Fig.  2.  Set  of  images  of  composite  actuators  consisting  of  a  stack  of  Si02/Si3N4/Si02/Ti/Pt/PZT/Pt.  Film  thicknesses  for  the  elastic  layer  are  indicated 
on  the  figures  while  the  Pt  (100  nm)  and  PZT  (500  nm)  are  nearly  constant  for  each  sample.  Micrographs  show  where,  (a)  the  Si3N4  is  closest  to  the 
bottom  interface,  (b)  the  S^'N4  thin  film  is  in  the  middle  of  the  Si02/SiiN4/Si02  composite,  (c)  the        is  closest  to  the  Pt  bottom  electrode. 
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Fig.  3.  Process  flow  schematic  for  PiezoMEMS  devices  using  the 
substrate  configurations  highUghted  in  Fig.  1.  Examples  of  possible 
XeF2,  Deep  Reactive  Ion  Etching,  Hydrofluoric  Acid  Released  cross- 
sections  are  shown. 
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Fig.  4.  Rocking  Curve-Full  Width  Half  Maximum  RC-FWHM 
showing  correlation  of  Pt  (222)  and  Ti02  (200)  orientation. 

for  PZT  growth  and  provides  the  basis  of  many  of  the 
devices  that  will  be  discussed  in  later  sections. 

The  details  regarding  the  Ti02/Pt  templates  with  an  elastic 
layer  of  500  nm  Si02  film  commercially  grown  by  thermal 
oxidation  on  Si  (100)  are  discussed  in  Ref.  17.  The  template 
begins  with  a  well-oriented  20  nm  thick  Ti  metal  layer  with  a 
rocking  curve  full-width  half  maximum  (RC-FWHM)  of 
approximately  4.5°.  Next,  the  Ti  is  converted  to  a  well-ori- 
ented Ti02  in  the  rutile  phase  with  a  RC-FWHM  of 
4.5  ±  1.5°  degrees  by  a  furnace  anneal  at  750°C  in  flowing 
oxygen.  Finally,  a  Pt  film  is  sputter  deposited  at  500°C  yield- 
ing a  Pt  (111)  film  with  a  RC-FWHM  of  2  ±  0.5°.  The  X-ray 
diffraction  RC-FWHM  plotted  in  Fig.  4  shows  that  the  ori- 
entation of  the  Pt  correlates  well  with  that  of  the  Ti02,  lower 
(i.e.,  improved)  Pt  RC-FWHM  being  obtained  for  lower 
Ti02  RC-FWHM.  This  comparison  demonstrates  the  causal 
impact  of  Ti02  orientation  improvement  on  that  of  Pt.  The 
influence  of  the  Pt  quahty  on  the  texturing  of  PZT  films  with 
a  Zr/Ti  ratio  of  52/48  is  illustrated  in  Fig.  5.  In  this  compari- 
son, the  PZT  films  were  deposited  via  chemical  solution 
deposition  (described  in  greater  detail  in  the  next  section)  on 
a  highly  textured  Ti02/Pt  template  and  an  un-textured  Ti/Pt 
template.  The  results  from  Fig.  5  show  that  a  properly  ori- 
ented Ti02/Pt  layer  provides  better  PZT  (111)  peaks  with  a 
Lotgering  Factor /(ni)  ~  0.95  compared  with  that  of  Ti/Pt 
with  a  /(111)  ~  0.45. 

As  mentioned  in  the  previous  section,  plasma  enhanced 
chemical  vapor  deposition  (PECVD)  Si3N4  and  Si02  multi- 
layer composites  can  be  used  to  control  the  stress  in  the  elas- 
tic layer  for  fine  tuning  the  residual  stress  induced 
deformation  in  PiezoMEMS  composites.  However,  the  as- 
deposited  surface  of  the  PECVD  films  was  found  to  have  an 
unacceptably  high  surface  roughness  compared  with  the  ther- 
maUy  grown  Si02  elastic  layer  used  in  the  work  described  in 
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Fig.  5.  X-ray  diffraction  scans  for  (a)  PZT/Pt/Ti  and  (b)  PZT/Pt/ 
Ti02  for  Zr/Ti  =  0.52/0.48.  AH  PZT  films  were  prepared  via 
chemical  solution  deposition. 


Table  I.    Thickness,  Sheet  Resistance  {R^^  and  Surface 
Roughness  Calculated  from  AFM  Measurements  on  Elastic 
Layers  of  Thermal  SiOi,  As-deposited  SiOi,  and  Chemical 

Mechanical  Polished  (CMP)  Si02.  In  Addition,  XRD 
Measurement  of  Rocking  Curve-Full  Width  Half  Maximum 
(RC-FWHM)  on  Ti,  TiOi,  and  Pt  Thin  Fihns  on  the  Various 
Elastic  Layers  is  Presented 


Elastic  Layer  Type 


Thermal     PECVD  PECVD 
Test  Units       SiOa  SiOj       SiOj  After  CMP 


Elastic  Layer 


Surface  Roughness 

nm 

0.211 

2.8 

-4.2 

0.30-0.38 

Thickness 

|im 

0.5 

0.7 

-2.2 

0.5-2.0 

Titanium 

Jl/sq 

45 

81- 

-95 

50 

RC-FWHM 

deg 

4.5 

22- 

-27 

5.7 

Titanium  Oxide 

Thickness 

nm 

33 

33 

33 

Platinum 

Jl/sq 

1.26 

1.3 

1.26 

RC-FWHM 

deg 

1.7-2.3 

12- 

-14 

2.25-2.35 

the  previous  paragraph  on  Ti02/Pt  templates.  The  increased 
surface  roughness  degraded  the  quality  of  the  deposited  Ti  as 
well  as  the  resulting  Ti02  and  Pt.  To  improve  the  quahty  of 
the  PECVD  films,  chemical  mechanical  pohshing  (CMP)  can 
be  used  to  decrease  the  roughness  of  the  as-deposited  thin 
films  to  nearly  that  of  thermally  grown  Si02.  As  an  example, 
a  combination  of  Si02  and  Si02/Si3N4/Si02  (Oxide/Nitride/ 
Oxide)  composite  films  deposited  by  PECVD  on  Si(lOO) 
wafers  underwent  CMP  performed  by  Axus  Technology, 
Chandler,  AZ.  In  each  case,  200  nm  of  Si02  was  removed 
during  the  pohshing  process.  Final  thicknesses  based  on 
deposition  rate  and  spectroscopic  ellipsometry  measurements 
of  the  PECVD  Si02  and  oxide  and  nitride  composites  are 
given  in  Table  L  Ti  sputter  deposition,  oxidation  to  convert 
to  Ti02,  and  Pt  sputter  deposition  were  performed  exactly  as 
described  above  for  the  Ti02/Pt  on  thermal  Si02.  Test  results 
to  characterize  the  elastic  layer,  Ti,  Ti02,  and  Pt  for  the 
unpohshed  PECVD  material,  CMP-processed  PECVD  mate- 
rial, and  standard  thermal  Si02  are  provided  in  Table  L  The 
data  show  that  pohshed  PECVD  surfaces  with  surface  rough- 
ness comparable  to  that  of  thermal  Si02  can  be  achieved, 
reducing  the  surface  roughness  of  untreated  PECVD  surfaces 
by  an  order  of  magnitude.  Pt  RC-FWHM  deposited  on  the 
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Fig.  6.  Correlation  of  Rocking  Curve  Full  Width  Half  Maximum 
(FWHM)  of  Pt  (222)  and  Ti  (001)  orientation  for  PECVD  [without 
and  with  chemical  mechanical  polished  (CMP)]  and  thermal  Si02 
elastic  layer. 


CMP-treated  PECVD  is  similar  to  that  for  Pt  on  thermal 
Si02  (-2.3°).  Fig.  6  shows  that  the  RC-FWHM  of  the  Ti  and 
Pt  films  are  decreased  in  magnitude  (improved)  by  the  CMP 
compared  with  the  case  for  no  CMP  treatment  of  the  PEC- 
VD elastic  layer.  Therefore,  the  data  show  that  Ti02/Pt  tem- 
plates obtained  on  CMP  PECVD  exhibit  satisfactory 
orientation  for  use  in  stress  engineered  PiezoMEMS  compos- 
ite structures. 


(3)    PZT  Optimization 

The  morphotropic  phase  boundary  (MPB)  of  ^h(Zrx  Tii.^) 
O3,  PZT,  exhibits  exceptional  piezoelectric  and  electrical 
properties  that  can  be  taken  advantage  of  in  MEMS  devices. 
At  the  MPB,  a  mixture  of  rhomobohedral  and  tetragonal 
phases  exists  resulting  in  an  increase  in  the  number  of  equiv- 
alent pohng  directions  (six  for  tetragonal  and  eight  for  rhom- 
bohedral)  and  exceptional  dielectric  and  piezoelectric 
properties.  These  properties  can  be  further  improved  by  con- 
trolling the  crystal  orientation  of  the  PZT  thin  films  to  be  in 
the  (001)  direction.  ^^"^^  For  the  following  discussions,  all  of 
the  PZT  films  were  deposited  with  a  Zr/Ti  ratio  of  52/48 
using  chemical  solution  deposition  (CSD)  from  a  solution 
with  a  2-Methoxyethanol  solvent  using  a  process  modified 
from  that  originally  described  by  Budd,  Dey,  and  Payne^^ 
and  Zhou,  Hong,  Wolf,  and  Trolier-McKinstry.^^  The  (001) 
crystal  orientation  in  PZT  (52/48)  films  prepared  by  the 
aforementioned  CSD  can  be  textured  with  proper  templating, 
annealing  procedures,  and  control  of  the  excess  Pb  content 
to  gain  higher  forces  and  displacements  and  lower  power 
consumption  in  PiezoMEMS  actuators.  Several  methods  of 
optimization  can  be  employed,  such  as  the  use  of  highly 
(111)  oriented  Pt  to  be  used  for  templating,  a  PbTiOs  seed 
layer  deposited  by  CSD,  and  thermal  treatment  of  PZT  and 
Pb-excess  to  achieve  highly  (001)/(100)  oriented  films  (for 
greater  detail  see  Ref.  24). 

The  use  of  the  PbTi03  seed  layer  is  inspired  by  the  work  of 
Paul  Muralt^^'^^  who  showed  that  the  PbTiOg  promoted  (100) 
growth  in  PZT  thin  films  and  suppressed  (111)  growth.  Simi- 
lar results  were  obtained  using  a  17  nm  thick  PbTi03  seed 
layer  deposited  by  CSD  in  conjunction  with  the  highly  tex- 
tured Ti02/Pt  template  (see  Fig.  7).  X-ray  diffraction  results 
using  a  Rigaku  Ultima  III  Diflfractometer  (The  Woodlands, 
TX)  with  Bragg-Brentano  optics  showed  /(ooi/ioo)  improving 
from  0.10  to  0.96  for  films  with  and  without  the  PbTi03  seed 
layer,  respectively.  Textured  PZT  using  the  PT  seed  layer 
exhibits  both  (001)  and  (100)  diffraction  indicating  the  pres- 
ence of  both  c-axis  and  a-Sixis  oriented  tetragonal  unit  cells. 


 :  >  . 

Fig.  7.  XRD  patterns  comparing  the  effects  of  using  the  PbTiOs 
seed  layer  on  highly  textured.  (Ill)  Pt  for  PZT  (52/48)  thin  films 
prepared  by  chemical  solution  deposition. 


Fig.  8.  Improvements  of  the  /(ooi/ioo)  are  observed  when  using  fast 
ramp  techniques  of  /(ooi/ioo)  =  0.99  compared  with  the  slow  ramp 
rate  with  /(ooi/ioo)  =  0-96  across  all  Pb-excess  PZT  (52/48) 
compositions  studied.  The  three  curves  represent  Ti02/Pt/PbTi03/ 
PZT  (52/48)  slow  ramp,  rapid  ramp,  and  double  ramp  data.  All  PZT 
films  were  prepared  via  chemical  solution  deposition. 

Further  improvements  to  the  (001)  orientation  of  PZT 
(52/48)  can  be  achieved  with  optimization  of  the  thermal 
treatment  and  Pb-excess  processed  in  the  films.  During  the 
crystallization  anneal  of  the  PZT,  volatile  PbO  is  lost  due  to 
high  temperature  processing.  This  PbO  loss  will  greatly  influ- 
ence not  only  the  nucleation  and  growth  characteristic  of 
PZT  but  will  also  lead  to  nonstoichiometric  PZT  and 
reduced  electrical  properties. Traditionally,  PZT  (52/48) 
films  were  annealed  at  700°C  for  60  s  in  a  rapid  thermal 
anneal  (RTA)  with  a  slow  ramp  (SR)  rate  of  4°C/s.  There 
are  several  theories  on  how  "slow"  or  "fast"  ramps  generate 
(001)  PZT  films. ^^"^^  Examination  of  two  alternative  anneal- 
ing conditions  using  fast  ramps  of  200°C/s  has  been  com- 
pared with  the  slow  ramp  of  4°C/s.  The  first  fast  ramp 
condition  involved  a  hold  time  of  60  s  at  700°C  which  was 
referred  to  as  rapid  ramp  (RR).  The  second  condition  was  a 
double  ramp  (DR)  technique  where  the  temperature  was 
ramped  to  550°C  to  minimize  Pb-loss  and  begin  nucleation 
events  at  a  lower  temperature.  After  60  s  the  temperature 
was  ramped  up  to  700°C  for  30  s  to  promote  perovskite 
growth.  The  optimization  of  the  PZT  (001)  texture  was 
found  to  occur  using  either  the  rapid  ramp  or  double  ramp 
in  conjunction  with  10%  excess  lead  (see  Fig.  8)  enabhng 
further  refinement  of  /(ooi/ioo)  to  0.99. 
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III.    Robotics  MEMS  Devices 

The  research  in  material  optimization  for  higher  PZT  piezo- 
electric coefficients  greatly  improves  performance  of  PZT 
sensors  and  actuators.  The  next  three  sections  highlight  how 
better  piezoelectric  materials  have  been  utilized  to  generate 
larger  forces,  displacements  and  greater  sensitivity  for  reduc- 
tions in  size,  weight,  and  power  (SWaP)  for  millimeter-scale 
robotic  appHcations. 


(1 )    PZT  MEMS  Actuators  for  Robotic  Mobility 

The  inherent  work  and  power  densities  of  PZT  thin  films 
enable  altogether  new  application  spaces.  One  example  is  the 
emerging  field  of  micro-  and  millimeter-scale  robotics. ^^'^^ 
Biologically  inspired  autonomous  millimeter-scale  robots 
have  a  range  of  anticipated  applications  resulting  from  their 
unique  scale,  low-cost,  and  potential  for  highly  integrated 
manufacturing.  The  large  electromechanical  coupling  factors 
and  piezoelectric  coefficients  of  PZT  thin  films  combined 
with  the  ability  to  monoUthically  couple  thin  film  actuators 
to  high  performance  flexural  and  structural  elements  allows 
PiezoMEMS  devices  to  meet  mobility  and  power  consump- 
tion system  needs  for  a  millimeter-scale  robotics  platform. 
Flexural  amplification  using  larger  load  bearing  structures 
has  demonstrated  the  abiUty  of  PZT  thin  film  actuators  to 
achieve  very  large  forces  and  displacements  with  small  foot- 
prints, low  power,  and  low  voltages. Thin  film  PZT  actua- 
tors could  avoid  the  inefficient  and  bulky  power  conditioning 
circuits  required  for  high  voltage  drive  and  permit  the  use  of 
the  monoUthically  integrated  PiezoMEMS  power  transform- 
ers discussed  in  a  later  section. Piezoelectric  sensing, 
described  in  the  following  section,  can  also  provide  efficient 
and  typically  passive  sensing  capabiUty  and  piezoelectric 
transduction  may  also  be  used  for  energy  harvesting.  The 
integration  of  the  other  complementary  PiezoMEMS  device 
technologies  described  in  this  article  provides  additional 
opportunities  for  achieving  highly  integrated,  capable,  and 
low  cost  millimeter-scale  robots. 

Integration  will  be  a  key  element  in  the  development  of 
these  small-scale  systems  by  mitigating  manufacturing  costs 
and  maximizing  system  capabiUties.  The  batch  fabrication  nat- 
ure of  MEMS  is  expected  to  permit  relatively  low  unit  costs 
and  limited  assembly  strategies  amenable  to  modular  system 
design.  Fig.  9  depicts  the  kind  of  notional  systems  that 
PiezoMEMS  technology  makes  possible.  Everything  for  the 
mobiUty  component  of  the  system  would  be  manufactured  in  a 
monolithic  PZT  MEMS  process,  described  above,  that  would 
provide  at  a  minimum,  all  of  the  necessary  components  like 


Fig.  9.  Illustration  of  a  notional  PiezoMEMS  enabled  multi-chip 
millimeter-scale  robot. 


joints  and  actuators  for  a  mobile  millimeter-scale  system.  The 
highUghted  chips  represent  the  other  requisite  subsystems,  hke 
power  or  control  that  are  required  for  a  robotic  platform.  The 
entire  system  is  expected  to  be  comprised  of  five  or  six  individ- 
ual chips,  manufactured  in  their  own  performance  and  cost 
optimal  processes,  and  simply  assembled  with  standard  IC 
packaging  techniques  for  a  complete  semi-autonomous  or 
autonomous  platform.  Such  an  approach  allows  PZT  MEMS 
device  and  fabrication  technologies  to  provide  the  unprece- 
dented ease  of  implementing  large  degree  of  freedom  robotic 
mechanisms;  the  strong  potential  for  simple  and  low-cost  inte- 
gration of  actuation,  sensing,  and  control;  and  the  relative  ease 
of  diverse  device  technology  integration. 

Actuation  technologies  intended  to  compete  with  their  bio- 
logical counterparts  at  this  scale  generally  must  achieve  large 
force  and  displacement,  extremely  low-power  consumption, 
provide  system  and  payload  enabUng  load  bearing  capacity, 
low  voltage  operation,  and  simpHcity  of  design  that  facih- 
tates  implementation  in  complex  robotic  mechanisms.  ARL 
has  developed  several  integrated  piezoelectric  lateral,  rota- 
tional, and  vertical  MEMS  actuators  that  meet  these  require- 
ments. Most  of  these  designs  utilize  the  <i3i  mode  of 
operation  and  to  varying  degrees  incorporate  the  unimorph 
bender  functionaUty.  Traditional  vertical  unimorph  actuators 
and  out-of-plane  rotational/torsional  actuators  can  provide 
leg  hfting  actuation,  see  Fig.  10.  Lateral  piezoelectric  actua- 
tors can  drive  in-plane  rotational  joints  whereas  coupled  high 
aspect  ratio  sihcon  or  metal  based  flexures  largely  support 
the  weight  of  the  robot. The  lateral  piezoelectric  actuators 
demonstrated  in  Ref.  34  provide  superior  force  and  displace- 
ment per  unit  power  over  rival  technologies,  and  are  the 
basis  for  essential  in-plane  pseudo-rotational  micro-robotic 
joints,  see  Fig.  10(a).  Due  to  inherent  nonlinearity,  these 
designs  produce  optimal  work  for  very  small  displacements 
(7  mN  at  1  jim  for  20  V  in  a  500  jim  x  100  |im  footprint  at 
~8  n/J  per  actuation)  and  thus  require  proper  flexural  ampU- 
fication  mechanism  design  to  translate  those  large  forces  into 
large  displacements.  The  individual  flexural  actuators,  piezo- 
electric actuators,  and  coupled  flexures,  can  provide  approxi- 
mately 5°  of  rotation  and  are  cascaded  to  create  individual 
joints  capable  of  significantly  larger  rotations.  Increasing 
load  bearing  capacity  and  hence  available  power  is  a  major 
focus  of  ongoing  research  in  actuator  performance,  flexure 
and  mechanism  process  design,  control  design,  and  joint/ 
robot  design. 

In  addition  to  terrestrial  mobiUty,  robotics  researchers 
are  attempting  to  create  highly  maneuverable,  small-scale 
vehicles  with  insect-inspired  flight  capabilities.  ^"^'^^  The 
ARL  is  attempting  to  reproduce  the  aerodynamic  forces 
generated  by  agile  insect  flyers  by  achieving  similar  kine- 
matic performance  in  comparable  millimeter-scale  wing 
structures.  Lift  production  in  insect  flight  depends  upon 
unsteady  aerodynamic  effects  associated  with  three-dimen- 
sional kinematics  utilizing  both  stroke  (i.e.,  flap)  and  pitch 
(i.e.,  wing  rotation)  degrees  of  freedom.  Previous  efforts 
demonstrated  the  feasibihty  of  achieving  insect-hke  stroke 
ampUtudes  in  thin-film  PZT  actuated  millimeter-scale 
wings^^  and  the  design  and  fabrication  of  the  integrated 
stroke  and  pitch  actuators  and  strain  sensors  were  reported 
at  Hilton  Head  2010.^^  The  2  mm  long  wings  are  of  a  sim- 
ilar scale  to  that  of  a  fruit-fly  (Drosophila).  The  actuated 
wing  design  features  two  independent  degree  of  freedom 
actuators  that  drive  a  micro-fabricated  wing  structure,  see 
Fig.  11.  The  thin  film  PZT  stroke  actuator  drives  the  flap- 
ping degree  of  freedom  at  resonance  and  the  pitch  actuator 
quasi-statically  drives  the  wing  rotation.  It  is  the  coordi- 
nated motion  of  these  two  degrees  of  freedom  and  the 
resulting  complex  three-dimensional  wing  kinematics  that 
generate  the  aerodynamic  hft  that  sustains  flight.  Such 
devices  have  successfully  demonstrated  120°  of  stroke 
ampUtude  at  10  V  and  45°  of  pitch  amplitude  at  20  V  at 
biologically  comparable  frequencies. 


Fig.  10.  Images  of  three  examples  of  PiezoMEMS  actuator  designs  applicable  to  millimeter-scale  robotics:  lateral  actuator  (left),  traditional 
vertical  unimorph  bender  (center),  and  in-plane  torsional  actuator  (right). 


Fig.  11.  (a)  Image  of  PiezoMEMS  actuated  wing  design,  illustrating  PZT  stroke  and  pitch  actuators  and  thin  film  wing  structure,  (b)  Image  of 
suspended  platform  with  individually  controllable  wings  next  to  a  US  dime. 


(2)    Proprioceptive  Sensing  with  thin  Film  PZT 

As  previously  described,  PZT  actuators  can  be  used  for 
mobility  in  small  robotic  systems.  Alternatively,  the  voltage 
generated  by  apphed  stresses  in  the  PZT  can  be  utilized  for 
inertial  sensing.  This  section  details  the  design,  fabrication, 
and  preliminary  results  of  a  biologically  inspired,  low-power, 
two-axis,  angular  rate  sensor  (ARS)  to  be  integrated  into 
micro-robotic  aerial  platforms.  Proprioceptive  sensors  enable 
mobihty  control  for  millimeter-scale  robotics  and  are  essen- 
tial to  achieving  stable  flight  in  aerial  autonomous  systems, 
but  they  are  challenging  to  integrate  at  this  scale.  For 
instance,  stable  hover  of  the  aforementioned  millimeter-scale 
flapping  wing  platforms  requires  three  types  of  propriocep- 
tive control. These  include  a  means  of  orientation  control, 
three-axis  relative  velocity  control,  and  three-axis  angular 
rate  control.  Most  flying  insects,  particularly  two  wing  or 
order  diptera,  use  both  visual  and  inertial  sensors  for  stable 
flight,^^  so  mimicking  biology  at  this  scale  requires  inertial 
sensors  also  at  a  similar  scale. 

Traditional,  packaged  rate  sensors  do  not  possess  compati- 
ble SWaP  to  be  practical  for  millimeter-scale  robotics.^  The 
mass  of  current  state  of  the  art  vibrating  MEMS  angular  rate 
sensors  is  individually  larger  than  the  desired  target  mass  of 
expected  entire  robotic  systems,  requiring  alternative  integra- 
tion strategies.  One  integration  option  is  to  eliminate  the  lead 
frames  and  plastic  encapsulation  then  die  bond  the  sensor  to 
the  robot.  This  non-traditional  packaging,  sensor  die  integra- 
tion, and  sensor  protection  creates  many  difficult  engineering 
issues  at  this  scale.  It  may  be  possible,  but  entails  packaging 
size  and  weight  trade-oflfs.  Ideally  the  rate  sensors  would  be 
made  monoHthically  in  the  process  that  makes  the  robotic 
platform  and  also  operate  effectively  in  air,  eliminating  the 
need   for   vacuum   packaging.    Wu   et  al.  demonstrated 


Fig.  12.  Example  of  the  club-like  haltere  location  (circled)  relative 
to  the  wings  of  diptera  on  a  Giant  Crane  Fly.  The  haltere  is  believed 
to  be  a  devolved  hind  wing  as  four  wing  insects  do  not  have  this 
structure.  Two  haltere  can  give  information  on  all  three-axes  of 
rotation. 


bio-mimetic,  ARS  that  can  detect  angular  rates  of  1  rad/s 
and  are  5-mm  in  length  driven  by  bulk  PZT  actuators. 
However,  these  are  not  small  enough  to  integrate  directly 
with  the  millimeter-scale  actuation  platforms  being  developed 
by  ARL,  but  they  provide  the  basis  for  scahng. 
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The  haltere  in  insects,  shown  in  Fig.  12,  measure  the  resul- 
tant CorioHs  forces  (caused  by  the  angular  rates)  and  motion 
orthogonal  to  the  direction  of  haltere  oscillations.  Force 
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Fig.  13.  Vectors  show  orthogonal  velocity  (v)  and  Coriolis  forces 
(Fc)  on  a  mass  due  to  angular  rate  ((d)  about  the  illustrated  axis 
vectors  on  a  notional  haltere.  The  orthogonal  aspect  ratios  of  the 
beam  allow  for  compliance  in  the  drive  and  sense  direction. 


Fig.  14.  SEM  of  a  PZT-based  MEMS  haltere  with  PZT  actuators 
to  enable  resonant  drive  and  PZT  sensors  along  the  cantilever 
section  to  detect  the  strains  generated  by  the  CorioHs  force  acting 
upon  the  vibrating  proof  mass. 


sensing  hairs  are  connected  to  their  neural  system  at  the  base 
of  the  haltere.  Inspired  by  the  insect  sensors,  a  MEMS-based 
pendulum  structure  that  is  both  actuated  and  sensed  piezo- 
electrically  has  been  created.  The  MEMS  ARS  are  designed 
to  be  monohthically  integrated  with  the  piezoelectric  mobihty 
actuators,  thus  enabhng  direct  proprioceptive  control  of 
millimeter-scale  and  cm-scale  robotic  platforms.  The  ARL 
PiezoMEMS  ARS  maximizes  the  sensitivity  to  rotational  rate 
(roll-pitch  or  yaw)  for  a  minimal  SWaP  requirement. 

For  a  MEMS  based  haltere.  Fig.  13  illustrates  the  general 
concept  with  the  Coriohs  forces  (Fc)  acting  on  the  end  mass 
due  to  angular  rate  about  the  illustrated  axis  vectors.  Thin 
film  based  versions  include  the  use  of  PZT  actuators  at  the 
base  of  the  structure  and  anchor  attachment  to  enable  reso- 
nant actuation.  PZT  sensing  sections  are  patterned  on  the 
Si02  layer  before  a  high  aspect  ratio  cantilever  section  is 
added  forming  an  inverted  "T"  cross-section.  The  PZT  con- 
verts lateral  Coriohs  force  generated  strain  in  the  film  to  a 
voltage  (see  Fig.  14).  Electroplated  metal,  Cu  in  this  case, 
was  used  to  fabricate  the  high  aspect  ratio  features  on  the 
cantilever  section.  The  out-of-plane  dimension  was  thickened 
to  reduce  the  sensitivity  of  the  sense  PZT  to  drive  motion. 
Figures  15,16  show  an  example  of  the  ANSYS  modeling 
used  to  predict  and  optimize  the  mode  shape  and  natural  fre- 
quencies of  the  drive  and  sense  modes  shapes,  respectively. 
The  haltere  was  designed  so  that  the  natural  frequency  of  the 
sense  mode  shape  is  at  least  2-3  x  higher  than  the  drive  fre- 
quency. This  is  to  ensure  that  the  haltere  sensors  can  respond 
to  the  frequency  of  the  Coriohs  force.  In  an  initial  fabrica- 
tion run  of  PZT/Cu  devices,  MEMS  haltere  has  demon- 
strated devices  vibrating  at  588  Hz  with  tip  displacements  of 
213  [im  for  a  500  |im  long  haltere  beam. 

Modehng  and  testing  indicates  the  area  of  the  PZT  desig- 
nated for  sensing  will  be  inherently  sensitive  to  the  Coriohs 
forces  caused  by  changes  in  angular  rate  with  a  1  rad/s 
design  goal.  Scale  studies  and  modal  analysis  were  completed 
to  guide  the  MEMS-level  sensor  development.  MEMS-based 
haltere  have  been  fabricated  as  individual  sensors  and  in  cou- 
pled arrays  in  10,  20,  and  30  micrometer  thick  Cu.  Under- 
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Fig.  15.  ANSYS  modal  analysis  showing  drive  or  actuation  mode  shape,  out  of  plane  motion  and  relative  strain  at  the  root  of  the  high  aspect 
ratio  beam  due  to  the  drive  motion.  (Note:  the  second  view  is  rotated) 


Fig.  16.  ANSYS  modal  analysis  showing  the  CorioHs  force  sensing  mode  shape,  in-plane  motion,  and  relative  strain  along  the  sides  of  the  high 
aspect  ratio  beam  due  to  the  sideways  motion.  (Note:  the  second  view  is  rotated) 


Fig.  17.  ANYS  modal  simulation  showing  the  mode  shape 
standing  wave.  A  traveHng  wave  results  from  two  standing  waves 
excited  90°  in  space  and  phase  relative  to  each  other.  The  patterned 
PZT  thin  film  process  enables  a  traveling  wave  by  ensuring  the 
antinodes  and  nodes  of  the  pair  of  standing  waves  are  aligned. 
Displacement  is  exaggerated  in  the  simulations.  Actual  motion  is 
<1  |im. 


standing  the  coupHng  mechanisms  between  multiple  haltere 
will  enable  triaxial-sensitivity  to  angular  rates  resulting  in 
reductions  in  size,  weight,  and  power  and  enabling  proprio- 
ceptive sensing  on  micro-air  vehicles  at  this  unprecedented 
scale. 


Fig.  18.  This  micrograph  of  a  fabricated  stator  2  mm  diameter 
stator  shows  the  electrode  layout  used  to  create  a  quarter-wavelength 
offset  in  space. 
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(3)    Traveling  Wave  Ultrasonic  Motors 

Many  small  scale  motors  have  been  developed  over  the  years, 
however,  few  if  any  are  able  to  produce  high  torques,  espe- 
cially at  low  speeds,  without  the  use  of  gearing.'^^''*^  TraveUng 
wave  ultrasonic  motors  (TWUM)  ofTer  a  path  toward 
addressing  this  need.  Advantages  of  TWUM  include  high 
torque-low  speed  performance,  silent  drive,  zero  power  hold- 
ing torque,  compact  design,  and  low  power."^^  Macroscale, 
bulk  PZT,  TWUM  have  found  wide  application  in  autofocus 
camera  lenses  as  well  as  automotive  appUcations.  Despite 
these  many  advantages,  miniaturization  of  TWUM  has 
stalled  due  to  the  minimum  resolution  of  the  manufacturing 
process.  The  PiezoMEMS  process  offers  many  advantages 
for  further  scaling  of  TWUM  including  high  resolution 
(<  1  |im),  batch  fabrication  enabhng  large  production  vol- 
umes, and  zero  hand  assembly. 

The  PiezoMEMS  based  TWUM  uses  a  thin  film  of  PZT 
to  generate  standing  and  traveUng  waves  within  the  PZT  and 
Si  stator.  The  stator  does  not  rotate,  but  the  elliptical  motion 
of  the  stator  surface  is  transferred  to  a  rotor  in  contact. 
Teeth  attached  to  the  stator  can  be  used  to  ampUfy  the  pie- 
zoelectrically  induced  elliptical  motion  in  the  stator  resulting 
in  increased  rotation  rates  in  the  motor.  ANSYS  simulations 
were  performed  to  optimize  the  performance  of  the  stator  for 
the  desired  modes.  The  B13  vibration  mode  is  shown  in 
Fig.  17  and  described  in  more  depth  in  Ref.  43.  In  addition, 
the  stators  were  designed  to  allow  for  control  of  the  location 
of  the  nodal  diameters  of  the  mode  shape  of  the  stator. 
Specifically,  the  top  Pt  electrode  (Act  Metal  1  from  Fig.  3) 
layout  was  configured  to  enhance  operation  of  the  B13  mode 
and  limit  the  number  of  external  interconnects  required  for 
operation  (see  Fig.  18). 

During  operation,  two  electrical  signals  were  used  to  cre- 
ate two  standing  waves  apart  in  space  and  offset  in  phase, 
which  add  by  superposition  to  create  a  traveUng  wave.  Direc- 
tional control  of  the  traveling  wave  was  accomphshed  by 
adjusting  relative  phase  of  the  two  drive  signals  from  +  90  to 
-  90  thereby  enabhng  bi-directional  rotation.  For  a  3  mm 


Fig.  19.    Micrograph  of  the  ultrasonic  motor  assembly. 


stator  using  30  |im  thick  Si  and  ~1  jim  thick  PZT  (52/48), 
the  resulting  traveling  wave  propagates  around  the  disc  every 
4  )is.  To  demonstrate  motor  rotation,  a  2  mm  wide  siUcon 
rotor,  manufactured  using  deep  silicon  reactive  ion  etching, 
was  placed  on  the  stator.  The  stator  was  then  wirebonded  to 
a  24-pin  dual  in-Une  package.  In  this  configuration,  the  wire- 
bonds  also  served  to  maintain  the  position  of  the  rotor  on 
the  stator.  The  assembled  device  is  shown  in  Figure  19. 
Upon  excitation  with  phase  offset  sine  wave  inputs  oscillating 
between  0  and  10  V,  the  rotor  spun  at  rates  ranging  from 
400  to  2300  rpm  depending  on  contact  with  the  wirebonds. 
When  rotational  direction  was  changed  by  altering  the  phase 
offset  between  the  two  drive  signals,  full  speed  change  of 
direction  occurred  within  30  ms.  This  assembly  was  used  as  a 
simple  proof  of  concept  for  a  PiezoMEMS  enabled  TWUM 
and  significant  performance  enhancement  is  expected  in  future 
integrated  designs  that  contain  integrated  confinement  and 
positioning  of  the  rotor  relative  to  the  stator. 

IV.    RF  MEMS  Devices 

In  the  area  of  RF  circuits,  RF  MEMS  technologies,  includ- 
ing switches  and  relays,  phase  shifters,  resonators,  filters,  and 
oscillators,  have  been  investigated.  These  technologies  can 
play  a  significant  role  in  reducing  loss,  power  consumption, 
improving  isolation  and  Unearity,  and  reducing  size  com- 
pared with  current  technology  solutions  for  tactical  and  com- 
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mercial  communication  systems  and  phased  array  radar  sys- 
tems. This  section  discusses  several  of  these  device  technolo- 
gies and  how  PiezoMEMS  technology  can  be  used  for 
additional  performance  benefits. 


(1 )    RF  MEMS  Switches 

Compared  with  other  RF  MEMS  technologies,  PZT  thin 
film  based  switches  offer  certain  benefits.  Usually,  the  low 
actuation  voltage  of  PZT  devices  is  pointed  out  as  their  sig- 
nature advantage."^"^'"^^  However,  PZT  devices  are  also  capa- 
ble of  generating  large  contact/restoring  forces  and 
displacements.  They  also  tend  to  consume  less  power  and  are 
typically  faster  than  thermally"^^  and  magnetically"^^  actuated 
RF  MEMS  devices.  When  compared  with  switches  actuated 
by  electrostatic  force,  an  additional  advantage  of  piezoelec- 
tric switches  is  the  full  control  of  their  displacement  when 
the  switch  is  making  contact.  Electrostatic  switches  are  char- 
acterized by  a  parameter  called  pull-down  voltage."^^  Once 
the  pull-down  voltage  is  reached  and  the  gap  of  the  electro- 
static capacitor  is  reduced  to  2/3  of  the  gap  at  0  V,  a  positive 
feedback  is  created  in  the  device.  A  smaller  gap  creates  larger 
closing  force,  which  in  turn  moves  the  switch  in  the  direction 
of  even  smaller  gaps.  This  process  of  constant  acceleration 
ends  when  the  RF  contact  is  closed  against  a  typically 
immovable  RF  line.  In  contrast,  PZT  RF  MEMS  switches 
move  only  in  response  to  the  magnitude  of  the  DC  actuation 
voltage.  They  do  not  exhibit  positive  feedback  and  their 
dynamic  behavior  is  well  controlled.  As  a  result,  switches 
with  piezoelectric  actuation  can  be  operated  where  the  con- 
tacts close  with  only  a  modest  acceleration  resulting  in  mini- 
mal contact  bounce  thus  improving  switch  cycle  rehabihty.'^^ 
Two  representative  PZT  based  switches  are  shown  in 
Fig.  20.  The  RF  circuit  is  reciprocal  using  a  co-planar  wave- 
guide (CPW)  transmission  hne  configuration  with  the  input 
or  output  on  the  left  or  right  hand  side  of  each  image.  The 
PZT  actuators  reside  between  the  center  conductor  and  the 
ground  plane  and  are  mechanically  coupled  with  the  elastic 
layer.  In  addition,  the  mechanical  link  provides  the  location 
for  the  electrical  contact,  which  completes  the  electrical  cir- 
cuit path  between  the  input  and  output  RF  transmission  line 
air  bridges.  At  zero  actuation  voltage,  the  input  and  output 
are  isolated.  Upon  applying  voltage,  the  actuators  move 
upward  out  of  the  plan  of  the  wafer  resulting  in  the  contact 
pad  contacting  both  air  bridges  and  completing  the  circuit 
between  the  input  and  output  circuit  path.  One  significant 
deviation  of  these  devices  from  the  earlier  pubhshed  results 
in  Refs.  45  and  46  is  the  use  of  a  4  \xm  thick  Si  elastic  layer 
from  a  SOI  wafer  as  opposed  to  a  0.5  |im  thick  composite 
silicon  dioxide  and  silicon  nitride  elastic  layer.  The  change  of 
elastic  layer  was  prompted  in  part  by  a  desire  for  devices 


with  an  increase  in  contact  force  and  increase  restoring  force 
for  improved  rehability  and  RF  performance  (for  more  infor- 
mation on  switch  reliabihty  see  reference  50).  Devices  with 
increasing  actuator  widths  are  shown  in  Fig.  20  to  demon- 
strate the  capabihties  of  the  fabrication  technology,  note  that 
release  holes  are  required  because  of  fabrication  tolerances. 
It  should  be  noted  that,  from  an  electrical  perspective,  these 
switches  are  four  terminal  devices  (i.e.,  the  two  terminals  for 
DC  actuation  and  the  two  RF  terminals  (input/ output)  are 
decoupled  from  each  other).  Such  a  topology  allows  indepen- 
dent optimization  of  the  mechanical  and  RF  performance  of 
the  devices. 

The  PZT  RF  MEMS  switches  were  characterized  using  an 
Agilent  E8361A  PNA  Network  Analyzer  in  the  frequency 
range  from  100  MHz  to  40.1  GHz.  In  the  off-state,  the  verti- 
cal gap  between  the  air  bridges  and  contact  pad  combined 
with  the  etch  trench  surrounding  the  contacts  enables  devices 
with  isolation  better  than  —20  dB  up  through  25  GHz  (see 
Figure  21).  The  devices  have  actuation  voltages  as  follows: 
single  actuator  -15  V;  dual  actuator  -20  V.  These  voltages 
are  2-2.5  times  higher  than  the  previously  pubhshed  results 
as  these  devices  have  yet  to  be  optimized  for  initial  gap  and 
temperature  stabihty.  Nevertheless,  the  contact  resistances  of 
these  switches  is  in  the  range  0.5-1  Ohm  with  the  corre- 
sponding insertion  loss  (see  Fig.  22)  better  than  —0.5  dB  up 
to  -15  GHz  (note,  insertion  loss  is  higher  than  it  should  be 
at  these  contact  resistances.  The  substrate  losses  were  sub- 
stantially higher  on  these  initial  devices  and  will  be  reduced 
on  subsequent  wafers). 


(2)    NanoMechanical  Logic 

Piezoelectric  RF  switches  have  also  become  enabling  in 
very  low  power  digital  logic  circuits.  The  low  leakage  current 
observed  on  a  PZT-based  actuator  has  led  to  the  develop- 
ment of  a  digital,  single-pole,  double-throw  (SPDT)  mechani- 
cal relay  with  zero  leakage  current  in  one  state  and  under 
30-fA  of  leakage  current  in  the  other  state. The  device  is 
shown  schematically  in  Fig.  23  and  with  the  two  independent 
current  paths  highhghted  in  green  in  Fig.  24.  In  addition  to 
low  leakage  currents,  the  structure's  material  composition 
provides  innate  radiation  hardening^^"^"^  and  the  SPDT  con- 
figuration of  this  device,  highhghted  in  Fig.  24,  makes  it 
optimally  suited  for  complimentary  logic  circuits. 

Due  to  PZT's  large  d2,\  coefficient,  large  deflections,  several 
hundred  nanometers  for  the  previously  mentioned  nanome- 
chanical  logic  relay,  can  be  achieved  at  modest  voltages.  This 
enables  the  relay  to  have  an  open-state  contact  gap  sufficient 
to  support  up  to  40  V  hot  switched  across  its  contacts,  whereas 
still  allowing  for  fully  functional  Boolean  logic  operations  at 
fewer  than  2  V  (see  Table  II).  In  addition,  due  to  the  ambipo- 


Fig.  20.    SEM  images  of  PZT-based  RF  MEMS  switches  with  actuators  of  different  width  (a)  single  actuator  and  (b)  dual  actuator. 
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Fig.  21.  Insertion  loss  in  the  on-state  (voltage  applied)  with  the 
single  actuator  in  red  and  dual  actuator  in  black. 
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Fig.  22.  Isolation  in  the  off-state  (0  V)  with  the  single  actuator  in 
red  and  dual  actuator  in  black. 
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Fig.  23.    Single-pole  double-throw  relay  schematic. 

lar  nature  of  metal-to -metal  contacts,  the  device  is  well  suited 
to  functioning  in  a  pass-gate  logic  configuration,  allowing  for 
a  reduction  in  gate  count  without  harming  the  signal  integrity. 
Fig.  25  demonstrates  this  behavior  with  a  Boolean  AND  oper- 
ation. Here,  the  top  electrode  of  Fig.  24  is  grounded,  the  bot- 


Fig.  24.  Electron  micrograph  of  a  fabricated  relay  highlighting  the 
two  paths  of  current  flow. 


tom  electrode  is  connected  to  "Input  A,"  the  normally  closed 
(NC)  input  is  grounded,  the  normally  open  (NO)  input  is  con- 
nected to  "Input  B"  and  both  the  NC  and  NO  outputs  are  con- 
nected together.  In  this  configuration,  when  "Input  A"  is  false, 
ground  is  shorted  to  the  output  and  when  "Input  A"  is  true, 
"Input  B"  is  passed  to  the  output. 

The  device  can  also  operate  as  either  of  the  two  classes  of 
digital  memory:  dynamic  and  static. The  low  leakage  of  the 
PZT  dielectric  makes  dynamic  memory  especially  attractive 
for  low  power  operation,  as  the  data  can  be  preserved  for  long 
periods  without  any  external  power.  Fig.  26  shows  one  relay, 
one  capacitor  implementation  of  a  dynamic  flip-flop.  This 
device  operates  by  propagating  "D"  to  the  storage  capacitor 
when  "PHI"  is  high  and  then  transporting  that  charge  to  "Q" 
as  soon  as  "PHI"  goes  low.  Measured  results  of  this  operation 
are  shown  in  Fig.  27.  Note  that  the  rapid  decay  shown  in  those 
results  is  due  to  the  1  M-Q  terminating  resistance  of  the  mea- 
surement equipment,  which  is  approximately  six  orders  of 
magnitude  lower  than  the  parasitic  resistance  of  the  MEMS 
relay,  the  component  this  circuit  is  intended  to  drive. 


(3 )    RF  MEMS  Resonators 

The  PZT  RF  MEMS  resonators  provide  a  potential  solution 
for  bandpass  and  bandstop  filters  in  communication  systems 
in  addition  to  high  quahty  factor,  low  phase  noise  oscillators 
for  timing  references  utilized  in  electronic  systems.  The  key 
advantage  for  contour-mode,  piezoelectric  MEMS  resonators 
is  the  multi-frequency  integration  on  a  single  chip.  These  res- 
onators also  show  promise  as  resonant  power  transformers 
for  single  chip  power  suppHes.^^  The  PZT  is  an  attractive 
choice  for  RF  MEMS  acoustic  resonators/transformers  due 
to  its  large  electromechanical  coupling  factors  and  stress  con- 
stants. The  high  electric  field  PZT  nonlinearity  also  allows  for 
electric  field  tuning  of  the  dielectric,  elastic,  piezoelectric  con- 
stants, and  couphng  factors;  enabhng  resonant  frequency  tun- 
ing, which  is  an  attractive  feature  for  RF  systems.  The 
approach  with  the  most  promise  to  date  is  to  combine  the 
high  electromechanical  coupling  of  the  PZT  material  with  the 
high  mechanical  quality  factor  of  sing  le  crystal  Si.^^  The 
silicon  layer  provides  a  means  for  higher  quality  factor,  lower 
loss  resonators.^''  PiezoMEMS  resonators  of  this  design  have 
recently  demonstrated  an  insertion  loss  as  low  as  2.1  dB  at 
~15  MHz  (see  Fig.  28),  a  loss  competitive  with  Film  Bulk 
Acoustic  Resonator  (FBAR)  technology.  Additional 
research  activities  in  thin  film  PZT  resonators  that  will  be  dis- 
cussed include  unique  tether  designs  that  minimize  anchor 
loss,  ultimately  improving  the  mechanical  quahty  factor,^^ 
tunable  parallel  resonance  with  >  110  dB  of  rejection  for 
notch  filter  apphcations  and  a  numerical  electrode  shaping 
design  technique  permitting  the  excitation  and  detection  of 
arbitrary  modes  in  arbitrary  geometries. In  addition,  these 
PZT  resonators  have  also  been  integrated  with  PZT  switches, 
enabling  a  monolithic  switchable  filter  approach. The  fol- 
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Table  II.    All  16  Boolean  Logic  Functions  Possible  with  PiezoMEMS  SPDT  Relay 


Boolean  Function 

Top  Pt 

Bottom  Pt 

NC  Input 

NC  Output             NO  Output 

NO  Input 

Always  0 

X 

X 

GROUND 

OUTPUT 

INPUT  B 

ANORB 

VDD 

INPUT  A 

GROUND 

OUTPUT 

NOT  INPUT  B 

(NOT  A)  AND  B 

GROUND 

INPUT  A 

INPUT  B 

OUTPUT 

GROUND 

NOT  A 

GROUND 

INPUT  A 

VDD 

OUTPUT 

GROUND 

A  AND  (NOT  B) 

GROUND 

INPUT  B 

INPUT  A 

OUTPUT 

GROUND 

NOT  B 

GROUND 

INPUT  B 

VDD 

OUTPUT 

GROUND 

A  XOR  R 

INPUT  A 

INPUT  R 

GR  OI IND 

OUTPUT 

VDD 

A NAND  B 

GROUND 

INPUT  A 

VDD 

OUTPUT 

NOT  INPUT  B 

A  AND  B 

GROUND 

INPUT  A 

GROUND 

OUTPUT 

INPUT  B 

A  XNOR  B 

INPUT  A 

INPUT  B 

VDD 

OUTPUT 

GROUND 

B 

X 

X 

INPUT  B 

OUTPUT 

INPUT  B 

(NOT  A)  OR  B 

GROUND 

INPUT  A 

VDD 

OUTPUT 

INPUT  B 

A 

X 

X 

INPUT  A 

OUTPUT 

INPUT  A 

A  OR  (NOT  B) 

GROUND 

INPUT  B 

VDD 

OUTPUT 

INPUT  A 

A  OR  B 

GROUND 

INPUT  A 

INPUT  B 

OUTPUT 

VDD 

Always  1 

X 

X 

VDD 

OUTPUT 

VDD 

Fig.  25.  Measured  results  of  Boolean  AND  function  being 
performed  via  a  PiezoMEMS  relay. 
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Fig.  26.    Optical  micrograph  of  a  dynamic  flip-flop. 


lowing  section  briefly  highlights  a  handful  of  current  efforts 
to  improve  performance  in  PiezoMEMS  resonator  devices. 

(A)  Anchor  Loss  Mitigation:  Anchor  loss  is  an  impor- 
tant research  topic  for  resonators  as  it  can  ultimately  limit 
the  mechanical  quality  factor  and  limit  device  performance. 
Quarter-wave  support  design  for  acoustic  matching  has  been 
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Fig.  27.  Measured  results  of  the  dynamic  flip-flop  created  from  two 
MEMS  relays.  Note  that  glitches  introduced  between  4.5  and  5  m\s 
do  not  propagate  to  the  output. 
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Fig.  28.  Insertion  loss  versus  frequency  for  a  PiezoMEMS  contour- 
mode  resonator  at  various  DC  bias  voltages.  At  10  Y  bias  a  low-loss 
of  2.1  dB  resulted. 

a  common  MEMS-based  approach  for  anchor  loss  mitiga- 
tion. In  general,  the  support  should  accommodate  large  reso- 
nator motion  and  velocity  at  the  attachment  point  while 
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simultaneously  providing  a  large  mechanical  impedance  at 
the  substrate  attachment;  ultimately,  an  efficient  impedance 
transformer  is  desired  where  acoustic  energy  radiating  into 
the  substrate  is  minimized.  An  alternative  design  involves 
determining  the  desired  resonator  geometric  and  resonant 
mode  and  then  subsequently  designing  a  tether  that  tends  to 
accommodate  the  general  resonator  motion.  Modal  analysis 
was  used  to  consider  dissimilar  anchor  support  designs  to 
gain  an  understanding  of  the  tether  response  (which  can  be 
considered  a  resonator  in  itself)  at  a  desired  resonant 
frequency.  The  Von  Mises  stresses  at  the  surface  between  the 
tether  and  the  substrate  is  then  used  to  assess  the  relative 
energy  that  would  be  radiated  when  comparing  two  variant 
tether  designs.  For  example,  two  tether  supports  are  consid- 
ered using  ANSYS  12.0  FEA  (Pittsburgh,  PA)  modal  analy- 
sis for  disc  resonator  geometries  as  shown  in  Fig.  29.  The 
tether  displacements  are  assessed  and  compared  (see  inset 
images  for  each  design  in  Fig.  29.  The  response  is  the  vector 
sum  of  the  unity  normalized  displacements,  where  the  blue 
represents  near  zero  relative  displacements.  It  is  evident  that 
the  alternative  tether  design  [Fig.  29(b)]  will  present  the  resona- 
tor with  reduced  energy  transfer  into  the  substrate  resulting  in 
a  larger  resonator  mechanical  quaUty  factor  and  improved 
device  performance. 

(B)  Electrode  Shaping:  Another  area  of  study  is  elec- 
trode shaping,  which  is  motivated  by  designing  for  the  most 
optimal  electrode  patterns  for  a  particular  resonant  mode 
to  both  minimize  insertion  loss  for  a  particular  resonant 
mode  and  improve  the  suppression  of  spurious  vibrational 
modes.  The  proper  design  of  the  resonator  electrodes  to 
selectively  excite  and  detect  desired  modes,  or  "electrode- 
shaping,"  can  provide  improvements  in  these  performance 
metrics.  Optimal  excitation  and  detection  electrodes  may  be 
analytically  derived  for  simple  one-dimensional  modes  by 
utilizing  the  orthogonality  property  of  the  vibrational  mode- 
shapes.  However,  extending  this  technique  to  more  compli- 


cated modes  is  difficult,  but  extending  the  theory  to  higher 
dimensional  modes,  such  as  those  existing  in  discs,  poses 
considerable  complications.  An  alternative  approach  has 
been  developed  that  determines  the  local  suitabihty  of  plac- 
ing an  electrode  on  the  resonator. The  local  suitability  is 
accompHshed  by  considering  strain  compatibiUty  between 
the  excited  piezoelectric  material  and  the  desired  mode  at 
each  location  on  the  resonator.  The  electrode  shapes  are 
ascertained  from  maps  of  this  local  determination  of  elec- 
trode placement.  For  excitation,  maximizing  the  local  strain 
leads  to  maximizing  the  local  contribution  to  the  lumped 
equivalent  excitation  force,  or  modal  force.  This  technique 
does  not  necessarily  maximally  excite  the  desired  mode  or 
maximally  suppress  unwanted  modes,  but  rather  determines 
electrode  shapes  that  ensure  the  desired  mode  is  excited  and 
detected.  Figure  30  displays  the  results  of  the  current  tech- 
nique for  the  detection  electrode  for  the  two-port  devices. 

(C)  PZT  Resonant  Power  Transformers:  An  alternate 
application  of  thin-film  PZT  resonators  includes  piezoelectric 
power  transformers  for  power  suppUes  in  on-chip  appUca- 
tions.^^  The  large  size  of  the  passive  components  limits  the 
size  of  the  switched  mode  power  supply  (SMPS),  thus  creat- 
ing the  need  for  a  high  frequency  SMPS  to  reduce  the  size  of 
the  passives.  Low  frequency  bulk  resonant  piezoelectric 
transformers  (<5  MHz)  have  been  implemented  in  the  past  in 
power  conditioning  units  for  advantages  including  high  volt- 
age isolation,  small  size,  and  the  absence  of  induced  electro- 
magnetic noise.  Thin  film  PZT  resonant  transformers  offer 
an  elegant  solution  for  on-chip  high  frequency  power  pas- 
sives integration. 

To  realize  either  a  voltage  buck  or  voltage  boost  configu- 
ration an  approximate  analysis  can  be  followed.  Assuming  a 
low  loss,  contour  mode,  two  port  piezoelectric  resonant 
transformer  as  depicted  in  Fig.  31,  the  input  power.  Pin,  is 
approximately  equal  to  the  output  power.  Based  on  this 
approximation. 


Fig.  29.  Modal  analysis  of  100  |im  diameter  PZT-on-Si  disc  resonator  operating  at  60  MHz  high  order  disc  flexure  mode.  The  plotted  response 
is  the  vector-sum  of  unity  normalized  displacements  (blue  identifies  near  zero  relative  displacements),  (left)  20  |im  long  x  5  |im  wide  straight 
tether  design  and  (right)  new  transformer  design. 


Fig.  30.  (Left)  FEA  modal  analysis  a  high  order  flexure  disc  mode  with  100  |im  diameter.  (Center)  Micrograph  of  the  fabricated  two-port  PZT- 
on-Si  disc  resonator  electrode-shaped  for  the  mode  illustrated  in  (left)  and  (right)  it's  corresponding  frequency  response.  The  dashed  red  line 
identifies  the  design  frequency. 
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Fig.  31.  Illustration  of  extensional  mode  resonator  where  the 
output  electrode  is  designed  based  on  the  resistive  load  to  which 
power  will  be  delivered.  The  output  electrode  area  is  traded  for  the 
input  transducer  area  for  voltage  boosting. 


Vout/V,n  =  /in//out  =  (^inMn)/(^outA^out) 


(1) 


(2) 


where  Fin  is  the  input  voltage  amphtude,  Fout  is  the  output 
amplitude,  /in  is  the  input  current,  /out  the  output  current,  A-^.^ 
and  ylout  the  area  of  the  input  and  output  electrodes,  respec- 
tively, and  TVin  and  #out  are  the  number  of  input  and  output 
electrodes,  respectively.  Therefore,  voltage  bucking  or  voltage 
boosting  occurs  with  mismatched  input  and  output  electrode 
areas.  A  detailed  analysis  of  electrode  area  trade-offs  for  the 
fundamental  length  extensional  modes  can  be  found  in^^  for  a 
thin  film  PZT  resonator  (single  PZT  stack).  The  trade  for 
input  area  versus  output  area  is  considered  for  a  device  illus- 
trated in  Fig.  31  with  a  190  )im  x  40  jim  x  11  jim  volume. 
The  voltage  gain,  efficiency,  and  power  delivery  are  all  a  func- 
tion of  the  input  and  output  transducer  design  areas  and  the 
resistive  load  value.  With  a  contour  mode  resonator  of  Q 
-2000,  voltage  boosts  as  high  as  8,  with  29%  efficiency,  are 
predicted.  In  addition,  efficiencies  as  high  as  80%  are  pre- 
dicted for  1:1  transformers  depicted  in  Fig.  31. 

As  previously  mentioned,  the  PZT  based  PiezoMEMS  reso- 
nators utilize  a  single  crystal  siUcon  elastic  layer  which  serves 
as  a  high  quahty  factor  resonator  material.  In  addition  to 
higher  quahty  factor  devices,  the  implement  of  silicon  signifi- 
cantly improves  the  device  power  handhng,^^  which  is  essential 
for  high  power  density  SMPS.  There  is,  however,  a  fundamen- 
tal trade  with  improved  quality  factor  and  power  handhng 
with  electromechanical  coupling  factor,  k^ff^.  Therefore,  the 
kes^  X  Q  metric  is  an  important  design  parameter  as  it  directly 
impacts  the  open  circuit  voltage  gain  and  efficiency. 


V.  Conclusion 

Over  the  last  decade,  the  potential  of  nano  to  millimeter  scale 
PZT  thin-film  devices  is  finally  being  realized  across  a  breath 
of  commercial  and  mihtary  areas.  Material,  design,  and  fabri- 
cation optimization  are  all  areas  that  have  fueled  this  growth. 
In  many  areas,  hke  FRAM,  ink-jet  printing,  RF  MEMS,  and 
millimeter-scale  actuation,  thin-film  PZT  transduction  has 
proven  itself  as  a  truly  enabhng  solution  to  engineering  chal- 
lenges that  have  been  studied  for  decades.  Compatible  pro- 
cesses and  integration  of  these  individual  PZT  solutions  will 
only  enhance  system  performance  while  reducing  components 
to  size,  weight,  and  power  scales  never  thought  possible. 


Acknowledgments 

The  authors  thank  their  many  collaborators  over  the  years  of  development  on 
PiezoMEMS  technology  at  the  US  Army  Research  Laboratory  including  the 


vision  of  Madan  Dubey  and  Robert  Zeto  in  initiating  the  PZT  MEMS 
research  effort.  Specifically,  Joel  Martin  and  Brian  Power  are  acknowledged 
for  their  contributions  to  device  fabrication.  In  addition,  the  roles  of  Fox 
Materials  Consulting,  Pennsylvania  State  University,  Cornell  University,  and 
the  University  of  Maryland  are  acknowledged  for  their  contributions  to  the 
fabrication,  design,  and  evaluation  of  PiezoMEMS  devices. 


References 

'R.  G.  Polcawich  and  J.  S.  Pulskamp,  "Chapter  5:  Additive  Processes  for 
Piezoelectric  Materials:Piezoelectric  MEMS;"  pp.  273-355  in  MEMS  Materials 
and  Processes  Handbook,  Edited  by  R.  Ghodssi  and  P.  Lin.  Springer,  NY, 
2011. 

^P.  Muralt,  R.  G.  Polcawich,  and  S.  Trolier-McKinstry,  "Piezoelectric  Thin 
Films  for  Sensors,  Actuators,  and  Energy  Harvesting,"  MRS  Bull.,  34,  658-64 
(2009). 

^S.  Tadigadapa  and  K.  Mateti,  "Piezoelectric  MEMS  Sensors:  State-of-the- 
Art  and  Perspecdves,"  Meas.  Sci.  TechnoL,  20,  092001  (2009). 

"^P.  Muralt,  "Recent  Progress  in  Materials  Issues  for  Piezoelectric  MEMS," 
/.  Am.  Ceram.  Soc,  91  [5]  1385-96  (2008). 

^http://www.ramtron.com/products/nonvolatile-memory/ 

^http://www.ti.com/product/msp430fr5728 

''http://www.fujitsu.com/us/services/edevices/microelectronics/memory/fram/ 
^http://industrial.panasonic.com/www-data/pdf/ARCOOOO/ ARC0000CE13.pdf 
^http://www.epson.com/cgi-bin/Store/Landing/InkTechPrinter.jsp 
'''http://www.dimatix.com/files/SAMBA-Technology-Backgrounder.pdf 
'^J.  S.  Pulskamp,  A.  Wickenden,  R.  Polcawich,  B.  Piekarski,  M.  Dubey, 
and  G.  Smith,  "Mitigation  of  Residual  Film  Stress  Deformation  in  Multilayer 
Microelectromechanical  Systems  Cantilever  Devices,"  /.  Vac.  Sci.  Technol.  B, 
Microelectron.  Process.  Phenom.,  21  [6]  2482-6  (2003). 

'^E.  Zakar,  R.  Polcawich,  M.  Dubey,  J.  Pulskamp,  B.  Piekarski,  J.  Conrad, 
and  R.  Piekarz,  "Stress  Analysis  of  SiOa/Ta/Pt/PZT/Pt  Stack  for  MEMS 
Application,"  Proc.  12th  IEEE  Int.  Symp.  Appl.  Ferroelectr.,  194-7  (2000). 

'^J.  S.  Cross,  K.  Kurihara,  and  H.  Haneda,  "Diffusion  of  Oxygen  Tracer 
Into  Deuterium-Gas-Baked  IrOVPb(Zr,Ti03)Pt  Capacitors  and  Pb(Zr,Ti)03Pt 
Films,"  /.  of  Appl.  Phys.,  98  [9]  094107,  5pp  (2005). 

A.  Bursill,  I.  M.  Reaney,  D.  P.  Vijay,  and  S.  B.  Desu,  "Comparison  of 
Lead  Zirconate  Titanate  Thin  Films  on  Ruthenium  Oxide  and  Platinum  Elec- 
trodes," J.  of  Appl.  Phys.,  75  [3]  1521-5  (1994). 

'^T.  S.  Moise,  S.  R.  Summerfelt,  and  K.  R.  Udayakumar,  "Method  of 
Forming  PZT  Ferroelectric  Capacitors  for  Integrated  Circuits";  US  Patent 
7,935,543,  2011. 

'^G.  L.  Smith,  R.  G.  Polcawich,  J.  S.  Pulskamp,  T.  Waggoner,  and  J.  Con- 
ley,  "Atomic  Layer  Deposited  Alumina  for  use  as  an  Etch  Barrier  Against 
Xenon  Difluoride  Etching";  pp.  194-7  in  Proceeding  of  2010  Solid  State  Sen- 
sor, Actuator  and  Microsystems  Workshop,  Transducers  Research  Foundation, 
Hilton  Head,  2010. 

'^D.  M.  Potrepka,  G.  R.  Fox,  L.  M.  Sanchez,  and  R.  G.  Polcawich, 
"Pt/Ti02  Growth  Templates  for  Enhanced  PZT  Films  and  MEMS  Devices," 
MRS  Online  Proc.  Library,  Material  Research  Society,  1299,  mrsfl0-1299-s04- 
02  (2011). 

'^F.  K.  Lotgering,  "Topotactical  Reactions  with  Ferrimagnetic  Oxides  Hav- 
ing Hexagonal  Crystal  Structures,"  J.  Inorg.  Nucl.  Chem.,  9  [2]  113  (1959). 

'^B.  Jaffe,  R.  S.  Roth,  and  S.  MarzuUo,  "Piezoelectric  Properties  of  Lead 
Zirconate-  Lead  Titanate  Solid  Solution  Ceramics,"  /.  of  Appl.  Phys.,  25  [6] 
809-10  (1954). 

^''P.  Muralt,  T.  Maeder,  L.  Sagalowicz,  S.  Hiboux,  S.  Scalese,  D.  Naumovic, 
R.  G.  Agostino,  N.  Xanthopoulos,  H.  J.  Mathieu,  L.  Patthey,  and  E.  L.  Bull- 
ock, "Texture  Control  of  PbTiOg  and  Pb(Zr,Ti)02  Thin  Films  With  TiOa 
Seeding,"  J.  of  Appl.  Phys.,  83,  3835  (1998). 

^'S.  Hiboux,  P.  Muralt,  and  N.  Setter,  "Orientation  and  Compostition 
Dependence  of  Piezoelectric-Dielectric  Properties  of  Sputtered  Pb(Zr^,Tii.;^) 
OgThin  Films,"  Proc.  MRS,  596,  499-504  (1999). 

^^K.  D.  Budd,  S.  K.  Dey,  and  D.  A.  Payne,  "Sol-gel  Processing  of  PbTiOg, 
PbZrOs,  PZT,  and  PLZT  Thin  Films,"  Br.  Ceram.  Proc,  36,  107-21  (1985). 

^^Q.  F.  Zhou,  E.  Hong,  R.  Wolf,  and  S.  TroUer-McKinstry,  "Dielectric 
and  Piezoelectric  Properties  of  PZT  52/48  Thick  Films  With  (100)  and  Ran- 
dom Crystallorgraphic  Orientation,"  Mat.  Res.  Soc.  Symp.,  655,  1171-6 
(2001). 

^^L.  M.  Sanchez,  D.  M.  Potrepka,  G.  R.  Fox,  I.  Takeuchi,  and  R.  G. 
Polcawich,  "Improving  PZT  Thin  Film  Texture  Through  Pt  Metallization 
and  Seed  Layers,"  MRS  Online  Proc.  Library,  1299,  mrsfl0-1299-s04-09 
(2011). 

^^N.  Ledermann,  P.  Muralt,  J.  Baborowski,  S.  Gentil,  K.  Mukati,  M.  Can- 
toni,  A.  Seifert,  and  N.  Setter,  "Textured,  Piezoelectric  Pb(Zr^.,  Tii_;^.)03  Thin 
Films  for  MEMS:  Integration,  Deposition,  and  Properties,"  Sens.  Actuators  A: 
Phys.,  105  [2]  162-70  (2003). 

^^J.  Chen,  K.  R.  Udayakumar,  K.  G.  Brooks,  and  L.  E.  Cross,  "Rapid 
Thermal  Annealing  of  Sol-gel  Derived  Lead  Zirconate  Titanate  Thin  Films," 
J.  of  Appl.  Phys.,  71  [9]  4465  (1992). 

^^H.  N.  Al-Shareef,  K.  R.  Bellur,  O.  Auciello,  X.  Chen,  and  A.  I.  Kingon, 
"Effect  of  Composition  and  Annealing  Conditions  on  the  Electrical  Properties 
of  Pb(Zr^Tii_^-)03  Thin  Films  Deposited  by  the  Sol-gel  Process,"  Thin  Solid 
Films,  252  [1]  38  (1994). 

2^C.-R.  Cho,  W.-J.  Lee,  B.-G.  Yu,  and  B.-W.  Kim,  "Dielectric  and  Ferroelec- 
tric Response  as  a  Function  of  Annealing  Temperature  and  Film  Thickness  of 
Sol-gel  Deposited  Pb(Zro  saTio  48)03  Thin  Film,"  /.  of  Appl.  Phys.,  86,  2700-12 
(1999). 


27 


^^S.  Hollar,  A.  Flynn,  S.  Bergbreiter,  and  K.  S.  J.  Pister,  "Robot  Leg 
Motion  in  a  Planarized-SOI,  Two-Layer  Poly-Si  Process,"  /.  Microelectro- 
mech.  Syst.,  14,  725-40  (2005). 

^°J.  S.  Pulskamp,  R.  G.  Polcawich,  and  K.  Oldham,  "Highly  Integrated 
Piezo-MEMS  Enabled  Millimeter-Scale  Robotics";  Vol.  6,  pp.  797-805  in 
Proceedings  of  ASM E  IDETC/MNS,  ASME,  San  Diego,  CA,  2009. 

^^K.  Oldham,  J.  Pulskamp,  R.  Polcawich,  and  M.  Dubey,  "Thin-Film  PZT 
Actuators  With  Extended  Stroke,"  /.  Microelectromech.  Syst.,  17  [4]  890-9 

(2008)  . 

^^^S.  S.  Bedair,  J.  Pulskamp,  B.  Morgan,  and  R.  Polcawich,  "Performance 
Model  of  Electrode  Tailored  Thin  Film  Piezoelectric  Transformers  for 
High  Frequency  Switched  Mode  Power  Supphes";  pp.  436-8  in  Proceedings  of  - 
PowerMEMS,  Transducers  Research  Foundation,  Washington,  DC, 
2009. 

^^^K.  Oldham,  J.  Pulskamp,  and  R.  Polcawich,  "Thin-Film  Piezoelectric 
Actuators  for  Bio-Inspired  Micro-Robotic  Applications";  Integr.  Ferroelectr., 
95  [1]  54-65  2007. 

-^^R.  S.  Fearing,  K.H.  Chiang,  M.H.  Dickinson,  D.L.  Pick,  M.  Sitti,  and  J. 
Yan,  "Wing  Transmission  for  a  Micromechanical  Flying  Insect";  Vol.  2,  pp. 
1509-16  in  Proceedings  of  IEEE  Int.  Conference  on  Robotics  and  Automation, 
IEEE,  San  Fransisco,  CA,  2000. 

^^R.  J.  Wood,  "The  First  Takeoff  of  a  Biologically  Inspired  At-Scale 
Robotic  Insect,"  IEEE  Trans,  on  Robotics,  24  [2]  341-7  (2008). 

^^J.  R.  Bronson,  J.S.  Pulskamp,  R.G.  Polcawich,  CM.  Kroninger,  and  E.D. 
Wetzel,  "PZT  MEMS  Actuated  Flapping  Wings  for  Insect-Inspired  Robotics"; 
pp.  1047-1050,  Proceedings  of  IEEE  Conference  on  MEMS,  IEEE,  Sorrento, 
Italy,  2009. 

-^^J.  S.  Pulskamp,  G.  L.  Smith,  R.  G.  Polcawich,  C.  M.  Kroninger,  and  E.  D. 
Wetzel,  "Two  Degree  of  Freedom  PZT  MEMS  Actuated  Flapping  Wings  With 
Integrated  Force  Sensing";  pp.  390-391  Proceedings  of  Solid  State  Sensor,  Actu- 
ator and  Microsystems  Workshop,  TRF,  Hilton  Head  Island,  SC,  2010. 

^^^W.  Wu,  R.  Wood,  and  R.  Fearing,  "Halteres  for  the  Micromechanical 
Flying  Insect,"  Proc.  IEEE  Int.  Conf.  Robot.  Autom.  p.  60,  2002. 

^^^G.  Frankel  and  J.  W.  S.  Pringle,  "Halteres  of  Flies  as  Gyroscopic  Organs 
of  Equilibrium,"  Nature,  141,  919-21  (1938). 

"^^L.  F.  Fan,  Y.  C.  Tai,  and  R.  S.  MuUer,  "IC-Processed  Electrostatic 
Micromotors,"  Sens.  Actuators,  20,  41-7  (1990). 

"^^R.  Legtenberg,  E.  Berenschot,  J.  Van  Baar,  and  M.  Elwenspoek,  "An 
Electrostatic  Lower  Stator  Axial-Gap  Polysilicon  Wobble  Motor  Part  I: 
Design  and  Modeling,"  /.  Microelectromech.  Syst.,  1  [1]  87-93  (1998). 

"^^K.  Uchino,  "Piezoelectric  Ultrasonic  Motors:  Overview,"  Smart  Mater. 
Struct.,  7,  273-85  (1998). 

"^^G.  L.  Smith,  R.  Q.  Rudy,  R.  G.  Polcawich,  and  D.  L.  Devoe,  "Integrated 
Thin  Film  Piezoelectric  Traveling  Wave  Ultrasonic  Motor,"  Sens.  Actuators 
A:  Phys.,  in  press  (2011). 

"^"^R.  G.  Polcawich,  J.  S.  Pulskamp,  D.  Judy,  P.  Ranade,  S.  Trolier-Mc-Kins- 
try,  and  M.  Dubay,  "Surface  Micromachined  Microelectromechanical  Ohmic 
Series  Switch  Using  Thin-Film  Piezoelectric  Actuators,"  Microwave  Theory 
and  Techniques,  IEEE  Transactions,  55  [12]  2642-54  (2007). 

"^^R.  G.  Polcawich,  D.  Judy,  J.  S.  Pulskamp,  S.  T.  McKinstry,  and  M. 
Dubey,  "Advances  in  Piezoelectrically  Actuated  RF  MEMS  Switches 
and  Phase  Shifters,"  IEEE  MTT-S  Microwave  Symposium,  2083-6  (2007). 

■^^M.  Daneshmand,  S.  Fouladi,  R.  R.  Mansour,  M.  Lisi,  and  T.  Stajcer, 
"Thermally  Actuated  Latching  RF  MEMS  Switch  and  Its  Characteristics," 
Microwave  Theory  and  Techniques,  IEEE  Transactions,  57  [12]  3229-38 

(2009)  . 


"^^M.  Glickman,  P.  Tseng,  P.  Harison,  J.  Niblock,  T.  Goldberg,  I.  B.  Judy, 
and  J.  W.  Judy,  "High  Performance  Lateral  Actuating  Magnetic  MEMS 
Switch,"  /.  Microelectromechanical  Systems.,  20  [4]  842-51  (2011). 

"^^G.  M.  Rebeiz  and  J.  B.  Muldavin,  "RF  MEMS  Switches  and  Switch  Cir- 
cuits," Microwave  Magazine,  IEEE.,  2  [4]  59-71  (2001). 

"^^B.  McCarthy,  G.  G.  Adams,  and  N.  E.  McGruer,  "A  Dynamic  Model, 
Including  Contact  Bounce,  of  an  Electrostatically  Actuated  Microswitch,"  /. 
MEMS,  11,  [3]  276-83  (2002). 

^'^J.  L.  Ebel,  D.  J.  Hyman,  and  H.  S.  Newman,  "RF  MEMS  Testing  - 
Beyond  the  S-Parameters,"  IEEE  Microwave  Mag.,  8,  76-88  (2007). 

^^R.  M.  Proie  Jr,  R.  G.  Polcawich,  J.  S.  Pulskamp,  T.  Ivanov,  and  M. 
Zaghloul,  "Development  of  a  PZT  MEMS  Switch  Architecture  for  Low-Power 
Digital  Applications,"  /.  Microelectromech.  Syst.,  20  [4]  1032-42  (2011). 

^^F.  Wang,  and  V.  D.  Agrawal,  "Single  Event  Upset:  An  Embedded  Tuto- 
rial"; pp.  429-34,  VLSI  Design,  21st  International  Conference  on,  IEEE, 
Hyderabad,  India,  2008. 

^^J.  M.  Benedetto,  R.  A.  Moore,  F.  B.  McLean,  P.  S.  Brody,  and  S.  K. 
Dey,  "The  Effect  of  Ionizing  Radiation  on  sol-gel  Ferroelectric  PZT  Capaci- 
tors," Nuclear  Science,  IEEE  Transactions  on,  37  [6]  1713-7  (1990). 

^"^S.  C.  Lee,  G.  Teowee,  R.  D.  Schrimpf,  D.  P.  I.  Birnie,  D.  R.  Uhlmann, 
and  K.  F.  Galloway,  "Total-Dose  Radiation  Effects  on  sol-gel  Derived 
PZT  Thin  Films,"  Nuclear  Science,  IEEE  Transactions  on,  39  [6]  2036^3 
(1992). 

^^J.  M.  Rabaey,  A.  Chandrakasan,  and  B.  Nikohc,  in  Digital  Integrated 
Circuits:  A  Design  Perspective,  2nd  ed..  Edited  by  C.  S.  Sodini.  United  States  of 
America:  Pearson  Education,  Inc.,  Upper  Saddle  River,  2003. 

^^H.  Chandrahahm,  S.  A.  Bhave,  R.  Polcawich,  J.  Pulskamp,  D.  Judy,  R. 
Kaul,  and  M.  Dubey,  "Influence  of  Sihcon  on  Quahty  Factor,  Motional 
Impedance  and  Tuning  Range  of  PZT-Transduced  Resonators";  pp.  360-3  in 
Proceedings  of  2008  Solid  State  Sensors,  Actuators  &  Microsystems  Workshop, 
Transducers  Research  Foundation,  Hilton  Head,  SC,  2008. 

^^J.  S.  Pulskamp,  S.  S.  Bedair,  R.  G.  Polcawich,  D.  Judy,  and  S.  Bhave, 
"Ferroelectric  PZT  RF  MEMS  Resonators";  pp.  1-6  in  IEEE  Frequency  Con- 
trol Symposium  2011  Proceedings,  IEEE,  San  Francisco,  CA,  2011. 

^^S.  S.  Bedair,  J.  S.  Pulskamp,  R.  G.  Polcawich,  D.  Judy,  A.  GiUon,  S. 
Bhave,  and  B.  Morgan,  "Low  Loss  Micromachined  Lead  Zirconate  Titanate, 
Contour  Mode  Resonator  With  50  Q  Termination,"  pp.  708-12  IEEE  Interna- 
tional Conference  on  Micro  Electro  Mechanical  Systems  Proc,  Paris,  France, 
2012. 

^"^S.  S.  Bedair,  D.  Judy,  J.  Pulskamp,  R.  G.  Polcawich,  A.  Gillon, 
E.  Hwang,  and  S.  Bhave,  "High  Rejection,  Tunable  ParaUel  Resonance  in 
Micromachined  Lead  Zirconate  Titanate  on  Silicon  Resonators,"  Appl.  Phys. 
Lett.,  99  [10]  103509,  3pp  (2011). 

^°J.  S.  Pulskamp,  S.  S.  Bedair,  R.  G.  Polcawich,  G.  L.  Smith,  J.  Martin,  B. 
Power,  and  S.  A.  Bhave,  "Electrode-Shaping  for  the  Excitation  and  Detection  of 
Arbitrary  Modes  in  Arbitrary  Geometries  in  Piezoelectric  Resonators,"  IEEE 
Transactions  Ultrasonics  Ferroelectrics  &  Frequency  Control,  (in  press)  (2012). 

^^J.  S.  Pulskamp,  D.  C.  Judy,  R.  G.  Polcawich,  R.  Kaul,  H.  Chandrahahm, 
and  S.  A.  Bhave,  "Monolithically  Integrated  PiezoMEMS  SP2T  Switch  and 
Contour-Mode  Filters";  pp.  900-3  in  IEEE  International  Conference  on  Micro 
Electro  Mechanical  Systems  Proc,  IEEE,  Sorrento,  Italy,  2009. 

^^A.  V.  Carazo,  "50  Years  of  Piezoelectric  Transformers.  Trends  in  the 
Technology";  pp.  33^4  in  Proceedings  of  Materials  Research  Society  Sympo- 
sium, Material  Research  Society,  Boston,  MA,  2003. 

^^R.  Polcawich,  J.  S.  Pulskamp,  S.  S.  Bedair,  G.  Smith,  R.  Kaul,  C.  Kronin- 
ger, E.  Wetzel,  H.  Chandrahahm,  and  S.  A.  Bhave,  "Integrated  PiezoMEMS 
Actuators  and  Sensors,"  in  IEEE  Sensors  2193-96  2010.  □ 


Gabriel  L.  Smith  received  B.S.  and 
M.S.  degrees  in  mechanical  engi- 
neering from  the  University  of 
Maryland,  College  Park,  in  1999  and 
2002,  respectively.  He  has  worked  in 
MEMS  Design  for  the  past  15  years 
with  the  U.S.  Naval  Surface  Warfare 
Systems,  U.S.  Army  Armaments 
Research  Dev.  Eng  Center,  and  U.S. 
Army  Research  Laboratory.  He  has 
developed  MEMS  devices  for  safe 
and  arm  devices  for  torpedoes  and 
gun-launched  munitions,  piezoelectric  sensors,  and  actuators 
for  millimeter-scale  robotic  systems,  and  piezoelectric  traveling 
wave  ultrasonic  motors.  He  has  also  fielded  inertial  sensors  in 
Army  systems  for  target  and  launch  detection.  He  currently 
holds  six  US  patents  with  three  patents  pending  and  has 
authored  seven  journal  and  conference  papers  on  MEMS 
devices. 


Jeffrey  S.  Pulskamp  received  the  B. 
S.  degree  in  mechanical  engineer- 
ing from  the  University  of  Mary- 
land, College  Park,  in  2000.  He  is 
currently  a  MEMS  Design  and 
Mechanical  Engineer  with  the  Micro 
and  Nano  Materials  and  Devices 
Branch,  U.S.  Army  Research  Labo- 
ratory, Adelphi,  MD.  His  current 
research  interests  include  RF 
MEMS  devices,  electromechanical 
design  and  modeling  of  MEMS,  and 
millimeter-scale  robotics.  He  is  currently  the  holder  of  eight 
patents  related  to  piezoelectric  MEMS  devices  and  has  an 
additional  five  patents  pending. 


28 


Luz  M.  Sanchez  received  her  B.A. 
degree  in  Physics  from  Hunter  Col- 
lege -City  University  of  New  York 
(CUNY),  New  York,  N.Y.  in  Janu- 
ary 2008.  As  of  2008,  she  is  pursu- 
ing her  Ph.D.  in  Materials  Science 
and  Engineering  at  the  University 
of  Maryland  College  Park  under 
the  mentorship  of  Dr.  IchiroTakeu- 
chi.  She  is  also  working  as  a  stu- 
dent intern  at  the  U.S.  Army 
Research  Laboratory  in  the  Micro 
and  Nano  Materials  and  Electronic  Devices  Branch.  Her 
current  research  interests  are  in  the  optimization  of  sol-gel 
deposited  PZT  and  highly  oriented  PZT  in  multilayer 
structures. 

Daniel  M.  Potrepka  is  a  Senior 
Research  Electronics  Engineer  at  the 
U.S.  Army  Research  Laboratory  in 
Adelphi,  Maryland.  He  received  his 
Ph.D.  in  Physics  in  1998  from  the 
University  of  Connecticut.  Dr.  Pot- 
repka was  a  Process  Engineer  at 
Mostek  Corporation,  an  Associate 
Member  of  the  Technical  Staff  at 
RCA  Laboratories,  and  an  Associ- 
ate Research  Scientist  at  United 
Technologies  Research  Center  between 
1979  and  1993.  As  a  National  Research  Council  postdoctoral 
fellow  he  studied  perovskite  materials  and  their  ferroelectric  behav- 
ior and  is  currently  involved  in  PiezoMEMS  process  and  materials 
research.  His  current  research  is  focused  on  a  crystoallographic 
study  of  PiezoMEMS  response  in  PZT  for  both  sputtered  and 
atomic  layer  deposited  thin  films  and  on  packaging  applications 
for  MEMS.  He  has  authored  or  coauthored  more  than  40  techni- 
cal papers  and  has  five  patents. 

Robert  M.  Proie  Jr.  received  the  B. 
S.  degree  in  electrical  engineering 
from  the  University  of  Pittsburgh, 
Pittsburgh,  PA,  in  2007,  the  M.S. 
degree  and  the  Ph.D.  degree  in 
computer  engineering  from  The 
George  Washington  University 
(GWU),  Washington,  DC,  in  2010 
and  2011,  respectively.  He  is  cur- 
rently an  electronics  engineer  at  the 
U.S.  Army  Research  Laboratory, 
Adelphi,  MD.  His  research  interests 
are  in  the  areas  of  low-power  electronics  and  MEMS  device 
design.  His  current  focus  is  on  piezoelectric  MEMS  design 
and  verification  for  both  RF  and  digital  applications. 

Tony  G.  Ivanov  received  the  M.S. 
and  Ph.D.  degrees  in  electrical  engi- 
neering from  the  University  of  Cen- 
tral Florida,  Orlando,  in  1994  and 
1997,  respectively.  From  1997  to 
2003,  he  was  with  Bell  Laboratories 
(AT&T,  Lucent  Technologies,  Ag- 
ere Systems  Holmdel,  NJ),  where 
he  worked  on  SiGe  and  BiCMOS 
technology  development.  Starting 
and  Development  organization,  and 
he  was  involved  with  CMOS  sih- 
con-on-insulator  and  RF  microelectromechanical  systems 
(MEMS)  technologies  for  wireless  products.  He  is  currently 
with  the  Sensors  and  Electron  Devices  Directorate,  U.S. 
Army  Research  Laboratory,  Adelphi,  MD.  His  research 


interests  cover  technologies  and  circuits  for  RF/millimeter- 
wave  applications,  with  emphasis  on  MEMS.  He  has 
authored  or  coauthored  more  than  40  journal  and  conference 
papers  and  is  the  holder  of  13  U.S.  patents.  Dr.  Ivanov  is  a 
founder  of  the  IEEE  Central  North  CaroUna  Chapter  of 
ED/MTT/SSC.  He  has  been  a  member  of  the  Technical  Pro- 
gram Committees  for  the  IEEE  MTT-S  International  Micro- 
wave Symposium  and  IEEE  Antennas  and  Propagation 
Society  International  Symposiums,  and  he  has  been  involved 
in  IEEE  in  various  capacities. 

Ryan  Q.  Rudy  received  the  B.S. 
degree  and  the  M.S.E.  degree  in 
mechanical  engineering  from  the 
University  of  Michigan,  Ann  Arbor, 
MI,  in  2009  and  2010,  respectively. 
Since  2010  he  has  been  part  of  the 
MEMS  and  Microfluidics  Labora- 
tory at  the  University  of  Maryland, 
College  Park,  where  he  is  pursuing 
the  Ph.D.  degree.  His  current 
research  interests  include  millimeter- 
scale  robotic  actuators  with  a  specific 
focus  on  rotary  traveling  wave  ultrasonic  motors. 


William  D.  Nothwang  joined  the 
Army  Research  Laboratory  from 
Clemson  University  in  the  Fall  of 
2001  as  a  ASEE  Research  Fellow. 
In  the  Spring  of  2003,  he  became  a 
Distinguished  Army  Scholar  within 
the  Army  Research  Laboratory. 
Dr.  Nothwang's  past  research  expe- 
rience has  focused  on  ultra-thin 
fihn  hetero-structures,  residual  stress, 
thin  fihn  deposition,  and  integration, 
and  specifically  on  the  role  of  resid- 
ual stress  on  complex  nanostructures  composed  of  many  layers 
of  dissimilar  materials.  For  research  in  these  areas,  he  was  one 
of  the  recipients  on  both  the  2005  Army  R&D  Award  and  the 
2007  Army  R&D  Award.  In  2009,  he  transferred  to  the 
RF  Division  of  the  Sensors  and  Electron  Devices  Directorate 
of  ARL  to  take  on  a  leadership  role  within  the  MAST  pro- 
gram and  to  start  his  new  research  program.  Dr.  Nothwang's 
current  research  endeavors  focus  on  cyber-physical  sensing  and 
control,  including  bio-mimetic  sensors  for  low  power,  small  size 
applications. 


Sarah  S.  Bedair  received  the  B.S. 
degree  in  Apphed  Sciences  & 
Computer  Engineering  from  the 
University  of  North  Carohna  - 
Chapel  Hill  (2002,  Magna  Cum 
Laude  with  Highest  Distinction). 
She  also  received  her  Ph.D.  and  M. 
S.  degrees  in  Electrical  &  Computer 
Engineering  from  the  Carnegie 
Mellon  University  (2004  and  2008, 
respectively)  where  her  thesis  work 
focused  on  CMOS  MEMS  based 
chemical  gravimetric  sensors  and  mechanical  oscillators. 
Dr.  Bedair  spent  one  year  (August  2008-June  2009)  as  an 
Oak  Ridge  Associated  University  Postdoctoral  Fellow  at  the 
U.S  Army  Research  Laboratory  (ARL)  in  Adelphi,  Mary- 
land. She  has  been  a  staff  researcher  in  the  Power  Compo- 
nents Branch  at  ARL  since  June  2009.  Her  research  is  on 
the  modeUng  and  fabrication  of  MEMS  based  power  devices. 


29 


specifically  for  single  chip  scale  power  conversion  and  RF 
electronics.  She  has  contributed  to  over  30  research  publica- 
tions and  is  co-inventor  on  4  patent  appHcations  or  invention 
disclosures  awaiting  patent  submission.  She  is  also  a  mem- 
ber of  IEEE.  Dr.  Bedair  has  received  fellowships  and  awards 
for  excellence  in  academics  and  research.  These  include  the 
Phillip  and  Marsha  Dowd-Institute  for  Complex  Engineered 
Systems  Fellowship  and  the  James  D.  Crawford  Award  at 
UNC-Chapel  Hill.  She  also  received  the  2010  Excellence  in 
Federal  Career  -Technical  &  Scientific  Rookie  of  the  Year 
Award  and  the  2009  ARL  Research  and  Development 
Achievement  Award. 

Christopher  D.  Meyer  received  the 
B.S.  and  M.S.  degrees  in  electrical 
engineering  from  the  University  of 
Florida,  Gainesville,  FL,  in  2006 
and  2009,  respectively.  Since  2006 
he  has  been  part  of  the  Interdisci- 
pUnary  Microsystems  Group  at  the 
University  of  Florida  while  pursu- 
ing the  Ph.D.  degree.  His  current 
research  interests  include  three- 
dimensional  microfabrication  tech- 
nologies,  hybrid  integration,  and 

magnetic  power  systems. 


Ronald  G.  Polcawich  received  the 
B.S.  degree  in  materials  science  and 
engineering  from  Carnegie  Mellon 
University,  Pittsburgh,  PA,  in  1997, 
and  the  M.S.  degree  in  materials 
and  the  Ph.D.  degree  in  materials 
science  and  engineering  from  Penn- 
sylvania State  University,  Univer- 
sity Park,  in  1999  and  2007, 
respectively.  He  is  currently  a  Staff 
Researcher  with  the  Micro  and 
Nano  Materials  and  Devices 
Branch,  U.S.  Army  Research  Laboratory,  Adelphi,  MD, 
where  he  is  also  currently  the  Team  Lead  for  the  RF  MEMS 
and  Millimeter-Scale  Robotics  programs.  The  current 
research  programs  include  switches  and  phase  shifters  for 
phased  array  antennas,  tunable  MEMS  resonators/filters  for 
secure  communication,  and  mobile  unattended  sensor  plat- 
forms. His  research  activities  include  materials  processing  of 
lead  zirconatetitanate  thin  films,  MEMS  fabrication,  piezo- 
electric MEMS,  RF  components,  MEMS  actuators,  and  mil- 
limeter-scale robotics.  He  is  currently  the  holder  of  four 
patents  and  has  authored  over  30  journal  articles  and  one 
book  chapter  on  fabrication  and  design  of  piezoelectric 
MEMS  devices.  Dr.  Polcawich  is  a  member  of  IEEE. 


Modeling  and  Optimal  Low-Power  On-Off 
Control  of  Thin-Film  Piezoelectric 
Rotational  Actuators 


Biji  Edamana,  Bongsu  Hahn,  Jeffery  S.  Pulskamp,  Ronald  G.  Polcawich 

and  Kenn  Oldham 

lEEE/ASME  Transactions  on  Mechatronics,  16(5)  (2011) 


Kiiinpliii^  tiniL^ 


Research@>4lC£ 


32 
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Abstract — A  novel  open-loop  minimal  energy  on-off  servo  sys- 
tem and  control  strategy  are  described  for  ensuring  specified  dis- 
placements from  new  microscale  piezoelectric  rotational  joints  un- 
der extremely  strict  power  budgets.  The  rotational  joints  are  driven 
by  thin-film  lead-zirconate-titanate  actuators  and  are  targeted  for 
use  in  autonomous  terrestrial  microrobots.  A  lumped-parameter, 
second-order  model  of  anticipated  joint  behavior  is  utilized  to  esti- 
mate the  natural  frequency  and  damping  ratio  of  the  robot  joints, 
which,  in  turn,  are  used  to  identify  necessary  sampfing  rates  and 
switching  drive  circuit  parameters  for  implementation  of  on-off 
control.  An  identified  model  of  leg  joint  behavior  is  then  used  to 
both  verify  lumped-parameter  modeling  and  to  optimize  on-off 
input  sequences  to  the  rotary  joint.  The  optimization  procedure  in- 
corporates energy  costs  from  both  switching  and  holding  an  input 
voltage  on  microactuators  that  behave  as  a  capacitive  load,  while 
ensuring  that  specified  final  states  of  a  dynamic  system  are  achieved 
at  a  specified  point  in  time.  Optimization  is  done  via  a  new  applica- 
tion of  binary  programming.  In  addition,  modest  robustness  of  the 
system  response  to  parameter  variation  can  be  produced  during 
control  sequence  generation.  Optimized  input  sequences  are  ap- 
plied to  both  macroscale  piezoelectric  actuators  and  to  prototype 
thin-film  piezoelectric  leg  joints,  and  show  that  specified  actuator 
motions  can  be  achieved  with  energy  consumption  of  less  than  5  /xj 
per  movement. 

Index  Terms — Integer  programming,  microactuators,  micro- 
electromechanical  devices,  on-off  control,  piezoelectric  devices, 
switched  systems. 


1.  Introduction 

THE  OPPORTUNITY  to  dramatically  reduce  the  size  of  ac- 
tuators and  sensors  through  microelectromechanical  sys- 
tem (MEMS)  technology  makes  possible  a  variety  of  miniature 
autonomous  devices,  such  as  unattended  sensor  nodes  or  mobile 
microrobots.  However,  in  autonomous  operation,  taking  full  ad- 
vantage of  the  small  size  of  MEMS  mechanisms  requires  that 
one  also  ensure  small  size  of  power  sources,  circuitry,  and  other 
subsystems  required  to  operate  the  central  MEMS  component. 
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Fig.  1.  Concept  drawing  of  an  autonomous  microrobot  based  on  thin-film 
piezoelectric  actuator  joint  arrays. 

One  opportunity  for  reducing  power  needs  of  MEMS  microac- 
tuators is  aggressive  reduction  in  power  needed  to  perform  con- 
trolled actuator  movements.  A  new  optimal  on-off  switching 
controller,  as  described  in  this  paper,  is  a  promising  method 
for  meeting  especially  strict  constraints  on  power  consumption. 
This  method  accommodates  both  efficient  actuator  drive  cir- 
cuitry and  minimizes  servo  system  energy  usage  to  accomphsh 
prescribed  movements. 

Especially  severe  power  constraints  are  faced  when  MEMS 
actuators  are  intended  to  actuate  autonomous  microrobots,  on 
the  scale  of  one  centimeter  or  smaller,  as  in  the  concept  drawing 
shown  in  Fig.  1.  While  a  few  microrobots  at  this  scale  have 
been  demonstrated  in  autonomous  or  tethered  operation,  their 
mobility  has  been  limited  by  the  need  to  either  carry  a  large- 
power  supply,  by  limited  actuation  force  at  low  power,  or  by 
the  need  to  operate  on  a  dedicated  substrate.  Several  previous 
robots  have  been  based  on  thermal  actuation,  but  the  large- 
power  consumption  of  thermal  actuators  permits  only  single 
degree  of  freedom,  short-displacement  leg  joints  in  order  to  carry 
large-battery  mass  [l]-[3].  In  contrast,  microrobots  based  on 
electrostatic  actuation  can  be  powered  with  much  smaller  power 
supplies,  but  electrostatic  actuators  require  very  large  areas  to 
produce  significant  force,  permitting  only  a  few  limbs  or  degrees 
of  freedom  per  robot;  alternatively,  electrostatic  microrobots 
may  depend  on  a  specialized  substrate  to  deliver  power,  limiting 
autonomy  to  small  areas  [4]-[6]. 

Thin-film  piezoelectric  actuators,  in  contrast,  are  capable  of 
delivering  much  larger  forces  than  electrostatic  actuators  with 
much  smaller  power  requirements  than  thermal  actuators,  or 
other  potential  robotic  actuators,  such  as  electromagnetic  ac- 
tuators or  shape-memory  alloys.  A  thin-film  lateral  actuator 
has  been  previously  demonstrated  capable  of  generating  up  to 
3  X  10~^  N-m  of  work  in  a  500  jim  x  100  fim  area,  and  this  pa- 
per may  be  leveraged  using  silicon  microstructures  to  generate 
rotational  motion  of  3°  or  more  from  a  single  such  actuator  at 
20  V  [7].  Operated  in  multiactuator  arrays,  these  actuators  can 
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generate  large-angle  rotations  of  microrobotic  leg  joints.  A  typ- 
ical array,  as  discussed  in  this  paper,  would  have  a  capacitance 
of  less  than  1  nF,  corresponding  to  intrinsic  energy  consumption 
of  less  than  1  /iJ  each  time  an  actuator  is  charged  to  an  operating 
voltage  on  the  order  of  20  to  30  V.  Furthermore,  integrated  with 
silicon  structures,  they  could  potentially  carry  between  5  and 
50  mg  payloads  per  leg  [8],  [9].  However,  even  with  state-of- 
the-art  thin-film  battery  or  solar  power  sources  [10],  [11],  the 
power  budget  per  leg  at  that  payload  capacity  is  on  the  order  of  a 
few  hundred  microwatts  to  a  few  milliwatts,  making  extremely 
low-power  servo  systems  a  necessity  for  microrobots  based  on 
these  actuators.  In  addition,  reduction  in  power  requirements  to 
control  piezoelectric  actuators  could  be  useful  to  slightly  larger 
miniature  robots  [12],  [13],  where  energy  constraints  are  not  as 
severe,  but  servo  power  reduction  still  allows  for  larger  payloads. 

Unfortunately,  controlling  the  motion  of  a  robotic  appendage, 
as  to  move  a  leg  to  a  specified  angle  in  a  given  step,  introduces 
significant  additional  power  consumption  beyond  that  of  the 
actuator  itself.  Existing  operational  ampHfiers  operating  in  the 
range  of  20  to  30  V  consume  at  least  400  fiW  in  quiescent  power, 
and  in  a  typical  analog  amplifier  circuit  for  piezoelectric  actua- 
tors, as  much  as  95%  of  the  power  used  by  the  actuator  may  be 
wasted  [14].  As  a  result,  switching  interface  circuits  between  a 
high- voltage  supply  and  the  piezoelectric  actuator  are  desirable, 
with  motion  control  typically  applied  using  pulsewidth  mod- 
ulation (PWM).  However,  PWM  requires  very  high  switching 
frequencies,  which  dramatically  increases  power  consumption 
when  driving  a  capacitive  load.  Charge  recovery,  such  as  that 
developed  by  Campolo  et  al.  [15],  is  one  method  for  reduc- 
ing this  energy,  use  is  through,  but  inductor  sizes  necessary  for 
effective  charge  recovery  can  be  excessively  large  and  energy 
losses  are  still  most  prevalent  during  switching,  although  these 
losses  are  reduced.Another  way  of  rotating  through  fixed  angles 
is  to  make  use  of  mechanical  stops  [16],  [17];  the  drawback  in 
this  method  is  that  there  is  only  one  possible  angle  of  rotation. 
Meanwhile,  power  consumption  of  sensor  circuitry  required 
for  feedback  also  greatly  exceeds  power  consumption  of  thin- 
film  piezoelectric  actuators,  such  that  open-loop  control  may  be 
necessary  to  meet  especially  strict  power  limits. 

Open-loop  on-off  control,  using  a  limited  number  of  switch- 
ing transitions  per  actuator  movement,  can  produce  regulated 
movements  with  extremely  small  energy  usage,  provided  that 
on-off  switching  times  are  chosen  carefully  and  a  reasonable 
model  of  the  system  is  available.  Previous  optimization  tech- 
niques for  on-off  control  schemes  have  minimized  the  time  to 
reach  a  desired  set  of  system  states  [18],  or  the  amount  of  time 
spent  with  an  "on"  input  to  reach  a  desired  set  of  states  at  a 
specified  time  [19],  [20],  but  these  approaches  do  not  account 
for  switching  costs,  such  as  energy  usage  to  charge  a  capacitive 
actuator.  Alternatively,  full-fledged  hybrid  system  models  can 
be  used  to  approach  on-off  control  design  [21],  [22],  but  this 
may  be  an  unnecessarily  complex  approach  when  only  two  input 
selections  are  available. 

This  paper  presents  a  model  for  a  piezoelectric  microrobotic 
leg  joint,  a  low-power  switching  circuit  for  control  of  the  joint, 
and  an  optimization  procedure  for  generating  minimum  energy 
on-off  input  sequences  to  direct  the  resulting  motion.  The  open- 


loop  optimal  on-off  switching  sequences  drive  a  set  of  initial 
states  to  a  desired  set  of  output  states,  while  minimizing  total 
energy  usage.  The  optimization  procedure  is  based  on  binary 
programming,  which  permits  a  simpler  optimization  procedure 
than  that  of  full-fledged  hybrid  systems  approaches,  but  allows 
switching  costs  to  be  incorporated  into  the  optimization  cost, 
unlike  previous  controllers  designed  specifically  for  on-off  se- 
quence optimization.  Applied  to  the  piezoelectric  microactuator 
arrays  in  a  novel  rotational  joint  configuration,  the  controller  can 
drive  comparatively  large  displacements  to  specified  angles  with 
just  a  few  on-off  switching  transitions  and  extremely  low-power 
consumption.  The  control  inputs  are  applied  to  the  piezoelectric 
actuators  through  a  low-power  switching  drive  circuit,  to  further 
reduce  system  power  consumption. 

Following  this  introduction.  Section  II  introduces  the  thin- 
film  piezoelectric  actuator  and  leg  joint  models.  Section  III  de- 
scribes the  design  of  a  low-power  switching  circuit  for  interfac- 
ing. Section  IV  describes  the  on-off  input  sequence  optimization 
procedure  and  a  simulational  study.  Section  VI  provides  exper- 
imental results  from  the  on-off  controller  using  a  low-power 
switching  drive  circuit.  Section  VII  discusses  implications  for 
future  low-power  microrobot  operation  and  concludes  the  paper. 

II.  Actuator  Design  and  Modeling 
A.  Individual  Flexure  and  Actuators 

The  large-angle,  rotational  joint  to  be  controlled  consists  of  a 
series  of  eight  elastic  silicon  flexures  between  rigid  silicon  links 
developed  at  the  U.S.  Army  Research  Laboratory,  as  shown  in 
Fig.  2.  Each  link  contains  a  thin-film  piezoelectric  actuator  that 
applies  a  bending  moment  to  one  of  the  silicon  flexures  via  a 
thin  silicon  tether.  Net  rotation  at  the  tip  of  the  joint  is  the  sum 
of  the  rotation  angles  of  each  flexure-actuator  pair. 

Each  flexure  in  the  array  is  a  thin  silicon  beam  that  experiences 
elastic  deflection  due  to  the  moment  generated  by  the  actuator 
force  offset  from  the  tip  of  the  actuator  by  distance  Lact,  as 
shown  in  Fig.  2(c2)  and  (c3).  The  rotational  stiffness  of  each 
link  is  given  by  the  equation 

Etwl 


where  E  is  the  elastic  modulus  of  silicon,  t  is  the  thickness 
of  the  silicon  flexures,  Wf  the  flexure  width,  and  Lj  h  the 
flexure  length.  Flexure  parameters  as  described  and  fabricated 
are  shown  in  Table  I.  While  the  nominal  flexure  thickness  was 
to  be  10  /im  in  the  legs  tested,  excessive  undercut  of  the  sili- 
con structures  during  deep  reactive  ion  etching  resulted  in  thin 
silicon  structures  (flexures  and  tethers)  having  a  thickness  of 
only  approximately  6  ^m,  leading  to  a  somewhat  less  stiff  joint 
structure  than  anticipated. 

The  piezoelectric  actuators  consist  of  bend-up  and  bend-down 
unimorph  segments  connected  in  series  to  produce  net  in-plane 
displacement  at  the  actuator  tip,  as  shown  in  Fig.  3.  The  bend-up 
unimorph  segments  consist  of  a  silicon  dioxide  base  layer  that 
sets  the  height  of  the  neutral  bending  axis  below  the  pizezoelec- 
tric  film,  a  bottom  platinum  electrode,  a  lead-zirconate-titanate 
(PZT)  piezoelectric  thin  film,  and  a  top  platinum  electrode.  The 
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Fig.  2.  Multiactuator  leg  joint,  (a)  Optical  photograph,  (b)  Schematic  drawing 
of  components,  (cl)  Link  dimensions.  (c2)  Force  and  displacement  at  actuator 
tip.  (c3)  Equivalent  bending  moment  model,  (d)  Mass  and  inertia  definitions  in 
deflected  state. 


TABLE  I 

Flexure  and  Actuator  Parameters 


Parameter 

Label 

Units 

Nominal 

Fabricated 

Value 

Value 

Elastic  Modulus  (Si) 

E 

GPa 

170 

170 

Device  Thickness 

t 

^tm 

10 

6 

Flexure  Width 

Wf 

fim 

7 

6.7 

Flexure  Length 

/^m 

120 

120 

Spring  Constant 

ke 

/xN/rad/s 

0.38 

0.23 

Actuator  Offset 

Lad 

/j,m 

12 

12 

Tether  Width 

Wt 

5 

4.7 

Tether  Length 

Lt 

292 

292 

Fig.  3 .  Schematic  diagram  and  optical  images  of  an  individual  thin-film  piezo- 
electric lateral  actuator,  alone  and  tethered  to  single  flexure. 


Fig.  4.  Force  versus  displacement  curve  of  thin-film  piezoelectric  actuators  at 
20  V,  nominal  model  from  [7]  and  with  effects  of  fabrication  limitations. 

bend-down  unimorph  segments  have  an  additional  gold  film 
added,  to  raise  the  neutral  bending  axis  of  the  structure  above 
the  midline  of  the  PZT  film  in  that  segment.  The  net  effect  is 
to  produce  in-plane  lateral  actuators  with  the  large-force  capac- 
ity of  piezoelectric  actuation  and  increased  stroke  length  due 
to  nonlinearity  of  the  bend-up  and  bend-down  structure.  A  full 
force-displacement  model  of  these  actuators  was  previously  de- 
scribed in  [7],  and  the  resulting  force-displacement  curve  for  a 
nominal  500-//m-long,  100-//m-wide  actuator  at  20  V  is  shown 
in  Fig.  4.  The  effective  electroactive  piezoelectric  strain  coef- 
ficient ((isi^eff )  for  these  actuators  at  20  V  was  approximately 
—60  pnW,  while  the  dielectric  coefficient  was  approximately 
230. 

Unfortunately,  while  the  nominal  force-displacement  of  the 
actuators,  as  designed  and  tested  in  the  previous  work,  would 
generate  greater  than  3°  of  rotation  at  each  link,  flaws  dur- 
ing the  microfabrication  process  greatly  reduced  the  force- 
displacement  capabilities  of  the  actuators  tested  here.  Namely, 
insufficient  undercut  of  the  silicon  layer  beneath  the  actuators 
reduced  their  length  to  only  about  440  yum,  and  residual  stress 
in  the  thin  film  layers,  particularly  the  gold  films  used  to  gener- 
ate bend-down  motion  at  the  outer  segments  of  the  actuator,  has 
the  effect  of  shifting  the  force-displacement  curve  away  from  its 
nominal  position.  When  the  observed  length  of  the  actuators  fol- 
lowing fabrication  and  estimated  residual  stress  levels  are  incor- 
porated in  the  actuator  models,  the  expected  force-displacement 
curve  diminishes,  as  shown  in  the  additional  curves  in  Fig.  4. 
While  this  decrease  in  performance  is  not  a  significant  factor  in 
verifying  controller  performance  and  power  consumption,  re- 
ducing negative  effects  of  residual  stress  and  better  regulating 
actuator  length  and  stiffness  will  be  necessary  to  achieve  truly 
large  range-of-motion  microrobotic  joints. 

Conversion  of  the  actuator  force  as  a  function  of  displacement 
to  the  moment  on  the  flexures  as  a  function  of  flexure  rotation 
depends,  first,  on  the  offset  between  the  actuator  tether  and  the 
elastic  flexure  in  each  link,  and  second,  on  the  stiffness  of  the 
tether  itself.  For  a  given  offset  Lact ,  the  resulting  moment  due 
to  the  actuation  force  Fact  is  given  by 

=Facti^act.  (2) 

The  static  lateral  displacement  at  the  tip  of  each  actuator  Xact 
is  dependent  on  the  force  Fact  that  the  actuator  is  applying, 
and  is  given  by  the  force-displacement  curve  for  the  thin-film 
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Fig.  5.  Static  applied  actuator  moment  at  20  V  versus  rotation  angle  at  silicon 
flexures,  based  on  nominal  actuator  design  and  taking  into  account  all  fabrication 
limitations. 


Fig.  6.    Sample  image  of  leg  joint  at  0  and  20  V. 

piezoelectric  actuators.  The  actuator  tip  displacement  is  also 
equal  to  displacement  due  to  rotation  of  the  leg  link  (Lact  times 
rotation  angle  for  the  link  to  which  it  attached)  plus  any  ax- 
ial stretching  in  the  tether  xt.  A  corresponding  static  rotation 
Oma^{M)  for  each  leg  link  can  then  be  estimated  from 


^max(Mr 


where  Xt  is  given  by 


Ewtt 


(3) 


(4) 


with  Lt  being  the  length  of  the  tether  and  wt  is  the  tether  width. 

This  rotation  must  match  that  of  the  flexures,  such  that  the 
actual  static  rotation  of  each  link  occurs  at  the  point,  where  the 
actuation  moment  as  a  function  of  rotation  intersects  the  ro- 
tary stiffness  curve  of  the  flexures,  as  shown  in  Fig.  5.  Had  the 
actuators  been  functioning  at  full-nominal  performance  as  dis- 
cussed with  respect  to  the  force-displacement  curve  in  Fig.  4, 
anticipated  rotation  angle  of  each  link  for  a  20  V  input  was 
to  be  approximately  3°  or  0.052  rad.  At  the  reduced  actuation 
capacity  of  the  actuators  as  formed,  we  find  rotation  angles  of 
approximately  1°  or  0.021  rad  per  link,  and  8°  per  leg  joint.  Ex- 
perimentally, a  rotation  of  7.5°  from  the  entire  joint  is  observed 
at  20  V,  with  a  sample  rotary  leg  joint  at  0  and  20  V  shown 
in  Fig.  6.  Again,  this  reduction  in  displacement  is  primarily  a 
result  of  underetching  of  silicon  along  the  actuator  length,  and 
residual  stress  prestressing  the  actuator  against  the  direction  of 
desired  motion. 


B.  Lumped-Parameter  Dynamics 

Although  on-off  input  sequence  optimization  is  ultimately 
performed  using  an  identified  model  of  system  dynamics,  an 
analytical  model  for  joint  dynamics  is  useful  for  both  design- 
ing leg  joint  design  and  specifying  reasonable  sampling  times 
and  switching  circuit  parameters  when  implementing  the  on-off 
controller.  While  the  large  deflection  leg  joints  consist  of  several 
flexures  and  rigid  links  in  series,  the  small  mass  of  the  intermedi- 
ate links  allows  actuator  dynamics  to  be  described  with  reason- 
able accuracy  using  a  lumped-parameter  second-order  model, 
particularly  for  small  displacement  of  approximately  10°  or  less, 
where  small  angle  sine  and  cosine  approximations  are  effective. 
The  resulting  second-order  model  has  the  form  as  follows: 


^tot^tot  +  ' 


+  hotO  =  Gu{t) 


(5) 


where  Jtot  is  the  total  rotary  inertia  of  the  system,  6tot  is  a 
rotational  damping  coefficient,  /ctot  is  the  equivalent  total  spring 
stiffness,  G  is  the  equivalent  actuation  moment  per  volt,  and  ^z(t) 
is  the  applied  input  voltage. 

The  input  to  the  system  is  treated  as  constant  when  voltage  is 
applied,  taken  as  the  moment  applied  by  the  actuator  when  the 
moment  is  zero,  while  the  shape  of  the  moment-rotation  curve 
is  accounted  for  in  an  equivalent  stiffness  of  the  actuator.  As  a 
result,  /ctot  is  estimated  from 


^tot 


(6) 


where  is  the  number  of  links  in  the  joint,  and  /cact  is  an 
equivalent  stiffness  of  a  single  actuator,  taken  from  a  linear  fit 
of  the  moment-rotation  curve  shown  in  Fig.  5  over  the  range  of 
anticipated  link  rotation. 

The  total  inertia  Jtot  is  calculated  by  treating  the  intermediate 
links  as  point  masses  mi  with  the  distance  from  the  center  of 
mass  of  ith  link  to  the  first  flexure  being  and  the  total 
inertia  of  the  final  leg  structure,  incorporating  both  leg  inertia 
-^les;  =  Jn  and  mass  mie 


TUN 


N-l 


Jtot  ~  ^  +  Jn  +  tunLI 


(7) 


Damping  coefficients  in  micromechanical  systems  are  typi- 
cally quite  difficult  to  estimate.  In  the  case  of  the  piezoelectric 
leg  joints  modeled  here,  the  leg  and  joint  move  just  a  few  mi- 
crometers above  the  surface  of  the  surrounding  silicon  wafer, 
or  substrate,  which  would  typically  result  in  viscous  drag  be- 
ing a  significant  source  of  damping.  In  addition,  experimental 
system  identification,  described  in  following  section,  suggested 
that  viscous  drag  might  be  a  likely  source  of  damping  due  to 
the  similarity  of  experimental  response  to  that  of  a  second-order 
linear  system.  As  a  result,  a  basic  estimate  of  damping  coeffi- 
cient was  obtained  by  integrating  the  moment  due  to  viscous 
drag  between  the  underside  of  the  joint  and  the  wafer  surface 
Mfj,  and  equating  it  to  an  effective  total  damping  coefficient 
btot,  using  the  integral 


pLn  , out 

Jo 


liw{r)r 
9 


{rO)dr  =  hotO 


(8) 
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TABLE  II 

Variables  for  Lumped-Parameter  Model 


Parameter 

Laoei 

Units 

Value 

Number  of  Links 

N 

g 

Ar'tiinfrvr  ^Hffrip«*j 

;/M/rflH/Q 

0.55 

Predicted  Stiffness 

hot 

yiiN/rad/s 

0.097 

Link  Mass 

mi -my 

1  1 

Leg  Mass 

772. 

MS 

17 

Leg  Inertia 

Jn 

/ig-mm^ 

15 

Link  1  Distance 

fim 

220 

Link  2  Distance 

Ll^ 

fj.m 

440 

Link  7  Distance 

Llj 

fj.m 

1540 

Link  8  (leg)  Inner  Radius 

in 

filYl 

1760 

Link  8  (leg)  CoM  Distance 

Ln 

fim 

2700 

Link  8  (leg)  Outer  Radius 

fim 

4300 

Predicted  Inertia 

Jtot 

49 

Viscosity 

mN-s/m^ 

0.02 

Leg  width 

Wleg 

fim 

90 

Joint  Gap  to  Substrate 

9i 

/im 

2-40 

Predicted  Damping 

hot 

/iN-/j.m/rad/s 

2-4 

where  r  is  the  radius  from  the  base  of  the  joint,  Ln^oui  is  the 
outermost  point  on  the  joint  and  leg  structure,  is  the  dynamic 
viscosity  of  air,  w{r)  is  the  width  of  the  joint  as  a  function  of 
radius  from  the  base,  g  is  the  gap  between  the  underside  of  the 
leg  and  the  ground,  and  rO  is  the  approximate  hnear  velocity  of 
points  along  the  leg.  This  type  of  drag  depends  heavily  on  the 
precise  gap  between  each  hnk  in  the  joint  and  the  substrate,  and 
for  a  system  of  smaller  links  and  one  long  leg  link  at  the  end  is 
approximately  equal  to 

where  Ai^i  is  the  planar  area  of  the  ith  link,  gi  is  the  gap 
between  the  ith  hnk  and  the  ground,  Ln;^  and  Ln, out  are  the 
distance  from  the  first  flexure  to  the  inner  and  outer  dimensions 
of  the  leg  (the  final,  A^th  link),  and  wi^g  is  the  width  of  the 
leg.  Unfortunately,  the  gaps  gi  are  difficult  to  predict,  being 
dependent  on  residual  stress  in  the  completed  legs,  although  it 
can  be  taken  as  having  an  approximately  linear  increase  along 
the  links,  from  approximately  0.5  fim  at  the  first  hnk  to  gaps 
between  0.5  and  40  /im  at  the  leg  link  itself. 

Values  of  the  parameters  used  to  predict  system  dynamics 
are  shown  in  Table  II.  The  estimated  total  spring  stiffness  and 
moment  of  inertia  correspond  to  a  natural  frequency  uun  of  ap- 
proximately 1880  rad/s,  and  damping  ratio  between  2%  and 
20%  depending  on  specific  gap  widths  due  to  residual  stress. 
These  estimates  are  useful  for  both  refining  the  design  of  future 
leg  joints,  and  anticipating  sampling  and  response  time  need 
for  driving  circuitry  and  controllers,  in  this  case  indicating  sam- 
pling time  for  discretization  on  the  order  of  100  /is  or  less,  and 
ideal  switching-circuit  response  time,  an  order  of  magnitude 
or  more  smaller  than  that.  Note  also  that  samphng  rate  refers 
to  the  rate  at  which  inputs  may  be  changed,  not  any  sensing 
measurements,  as  the  system  is  run  in  open  loop  under  the  fol- 
lowing control  scheme  in  order  to  conserve  energy.  In  practice, 
a  100  /iS  sampling  rate  was  used  for  microscale  actuator  ex- 
periments, with  the  switching  circuit  described  in  the  following 
having  a  response  time  of  less  than  2  /iS. 
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Fig.  7.  Step  response  of  the  MEMS  actuator,  which  was  used  for  system 
identification. 


C.  Experimental  System  Identification 

Exact  system  dynamics  were  measured  experimentally  using 
the  step  response  of  the  completed  leg  joints.  Joint  motion  was 
measured  by  filming  the  microscale  leg  joints  through  a  stere- 
oscope using  a  high-speed  camera  system.  Images  were  col- 
lected at  4000  frames  per  second,  and  angle  measurements  in 
each  frame  were  extracted  using  the  MATLAB  Image  Process- 
ing Toolbox.  The  measured  response  and  an  identified  second- 
order  model  response  are  shown  in  Fig.  7.  The  identified  natural 
frequency  was  1770  rad/s,  while  the  identified  damping  ratio 
was  5.7%. 

These  identified  parameters  fit  in  well  with  the  estimated 
response  of  the  system,  given  uncertainties  in  MEMS  process- 
ing accuracy,  and  indicate  the  utility  of  approximated  lump- 
parameter  modehng  of  the  piezoelectric  leg  joints,  although 
some  nonlinearity  in  the  system  step  response  appears  to  be 
present,  which  is  likely  a  result  of  nonlinear  piezoelectric  be- 
havior (discussed  in  the  following)  and  unmodeled  damping 
effects,  such  as  variation  in  gap  between  the  joint  and  the  sub- 
strate as  the  joint  rotates;  there  is  a  shght  upward  deflection  of 
the  actuator  (approximately  1-2°)  due  to  residual  stress  in  the 
thin  films  on  the  surface  of  the  actuator.  This,  and  etch  holes 
remaining  from  the  actuator  fabrication  process  mean  that  the 
gap  between  the  underside  of  the  actuators  and  the  substrate 
surface  is  not  entirely  constant,  as  is  assumed  to  estimate  the 
approximate  linear  damping  coefficient.  In  particular,  the  gap 
between  joint  and  substrate  is  larger  at  the  deflected  position, 
leading  to  a  lower  effective  damping  coefficient  near  the  final 
position  than  the  initial  position  during  a  step  displacement. 

In  addition,  changes  in  system  parameters  with  environmental 
conditions,  and  hysteresis  of  piezoelectric  materials  can  cause 
the  real  system  to  deviate  from  models  used  for  controller  de- 
velopment. For  example,  the  deflection  of  the  piezoelectric  ac- 
tuators is  sensitive  to  temperature,  and  there  is  also  a  modest 
hysteresis  in  the  piezoelectric  actuators,  as  shown  in  Fig.  16. 
We  will  discuss  methods  of  improving  robustness  of  the  actua- 
tor response  under  open  loop  on-off  control  when  uncertainty  is 
present  in  controller  design.  Namely,  we  treat  the  stiffness  and 
damping  of  the  system  as  having  approximately  10%  uncertainty 
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Fig.  8.    Switching  drive  circuit  with  pull-up  resistors  to  limit  leakage  current. 


due  to  nonlinearities,  hysteresis,  or  environmental  changes  and 
attempt  to  minimize  worst-case  error  in  final  position  under 
this  uncertainty.  These  uncertainty  bounds  may  be  increased  if 
larger  variation  is  anticipated,  but  with  corresponding  reduction 
in  accuracy;  systems  subject  to  large  parameter  variation  would 
require  some  use  of  feedback  if  improved  accuracy  is  required, 
although  this  would  increase  total  actuator  power  consumption. 

III.  Drive  Circuit  Design  and  Testing 
A.  Circuit  Design 

To  limit  switching  losses  in  the  interface  between  a  low- 
voltage  controller  and  the  comparatively  high- voltage  actuators, 
a  low  power  on-off  switching  circuit  was  designed  to  interface 
the  two  elements.  As  this  interface  can  cause  a  significant  por- 
tion of  the  switching  energy  losses  for  microscale  actuators, 
it  is  important  to  be  able  to  predict  and  measure  these  losses 
for  incorporation  into  controller  design,  as  well  as  to  improve 
performance  of  the  overall  system. 

The  on-off  switching  circuit  designed  for  the  thin-film  piezo- 
electric actuators  consists  of  a  CMOS  inverter  with  a  level 
shifter.  CMOS  inverters  are  a  commonly  used  switching  circuit 
configuration  in  the  integrated  circuit  design  area  for  reducing 
power  consumption  because  in  the  ideal  case,  there  is  no  static 
current  and  power  is  consumed  only  at  the  "on"  or  "off  tran- 
sition time.  However,  a  CMOS  inverter  alone  cannot  be  used 
directly  for  driving  piezoelectric  actuators.  While  a  piezoelec- 
tric actuator  should  be  driven  at  20  to  30  V  or  more,  most  IC 
circuits,  as  are  typically  used  to  implement  a  control  law,  oper- 
ate at  5,  3.3  V,  or  less.  Therefore,  a  level  shifter  based  on  the 
CMOS  inverter  was  designed  to  interface  between  a  high  volt- 
age and  a  conventional  IC  process  [23].  In  order  to  reduce  the 
power  consumption  of  the  circuit,  two  resistors  are  added  to  the 
basic-level  shifter.  The  conceptual  circuit  is  shown  in  Fig.  8. 

The  novel  addition  of  these  resistors  to  the  inverter  helps  to 
reduce  peak  leakage  current  during  the  switching  transitions. 
While  this  loss  is  typically  of  little  consequence  when  using 
larger  actuators,  it  can  be  a  substantial  portion  of  energy  con- 
sumption when  working  with  microscale  piezoelectric  actua- 


tors having  comparatively  small  capacitance.  The  total  energy 
consumption  E'tot  due  to  the  circuit  alone  for  a  single  cycle 
(charging  and  discharging  of  the  actuator)  can  be  derived  from 
the  energy  consumed  by  transistor  capacitance  Ec^  and  the 
energy  consumed  in  leakage  through  the  right  and  left  sides  of 
the  bridge  upon  switching  Ea  and  Eb 


Etot  —  Ec^  -i-  Ea  -\-  Eb 
with  the  respective  powers  being  calculated  from 


Ea 


2CmU^ 


{Umax  —  Vtp)RlC„ 


S 
Vtn 


KR2Vtn 

Cm  (?/max  "         "  Kp  log  (f/,nax/Vip)) 


(10) 


(11) 


(12) 


+ 


Kg 

Vtp)RiCmVtn 


Urn 


KR2V,r, 


(13) 


assuming  that  the  low  voltage  achieved  by  the  switching  cir- 
cuit is  zero,  and  where  Cm  is  the  capacitance  of  the  MOSFET 
transistors,  U\ow  is  the  low- voltage  input  level  of  the  controller, 
t^max  is  the  high- voltage  "on"  voltage  to  the  actuators,  Vtp  is 
the  threshold  voltage  of  the  PMOS  transistor  in  the  circuit,  V^n 
is  the  threshold  voltage  of  the  NMOS  transistor  in  the  circuit, 
Ri  is  the  pull-up  resistor  resistance  on  the  low- voltage  CMOS 
inverter,  R2  is  the  pull-up  resistor  resistance  of  the  high-voltage 
CMOS  inverter,  d  is  the  load  capacitance  of  the  piezoelectric 
actuators,  and  Kg  is  the  gain  coefficient  of  the  transistors. 

The  piezoelectric  joints  have  a  total  capacitance  of  approxi- 
mately 1 . 1  nF,  while  the  transistors  utiHzed  had  threshold  volt- 
ages of  1  V,  capacitance  of  0.23  nF,  and  gain  coefficient  of  1.99. 
Pull-up  resistors  of  100  and  1000  0,  respectively,  were  used 
to  balance  response  time  of  the  circuit  with  low-power  con- 
sumption, while  using  readily  available  surface  mount  resistors. 
These  parameters  correspond  to  a  projected  energy  consumption 
of  the  circuit  alone  of  1 .02  /iJ  per  charge  and  discharge  cycle, 
with  0.84  /iJ  lost  due  to  leakage  current  and  the  remainder  due 
to  capacitive  loading  of  the  CMOS  transistors'  gate  capacitance. 

IV.  Controller  Design 

A.  System  Dynamics  and  Constraints 

As  described  earlier,  the  dynamics  of  the  single  leg  are  mod- 
eled as  a  second-order  linear  system,  which  can  be  discretized 
and  represented  in  state  space  form  as  follows: 

X  {{k  +  l)Ts)  =  Adx{kTs)  +  BdUk  (14) 

y{kTs)  =  Cdx{kTs)  (15) 

where  the  states  of  the  system  are  the  angle  of  rotation  (^tot) 
and  the  angular  velocity  (^tot)  of  the  leg.  The  discrete  time 
state  matrix  is  Ad,  input  matrix  is  Bd,  and  output  matrix  is  Cd- 
There  are  two  important  constraints  on  the  system  when  using 
on-off  control.  The  first  is  that  the  inputs  Uk,k  —  1,2,.  ..,n 
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can  take  only  two  values,  namely  {0, 1}.  Second,  the  transi- 
tions between  input  values  can  take  place  only  at  the  sampling 
instants,  meaning  Uk  can  change  its  value  only  at  these  times. 
These  constraints  limit  the  reachable  subspace  of  the  system  in 
a  given  time  duration. 

B.  Problem  Statement  and  Optimization  Procedure 

The  unavailability  of  feedback  and  aforementioned  input  con- 
straints limits  the  capabilities  of  this  system.  So,  the  limited 
objective  of  a  controller  is  to  make  the  leg  joint  rotate  to  a  de- 
sired angle  and  stay  stationary  at  least  instantaneously  at  the 
end  of  a  desired  time,  which  will  enable  the  robot  to  walk  in  a 
quasi-static  manner. 

Mathematically,  then,  the  aim  here  is  to  find  a  sequence  of 
inputs  Uk,k  =  1,2, ...  ,n,  which,  if  a  solution  exists,  will  move 
the  states  of  the  system  from  a  specified  set  of  initial  states  to 
an  e  ball  containing  a  set  of  desired  final  states  Xd  in  a  given 
time  tf  =  nTg  (which  limits  the  possible  number  of  transitions 
between  the  input  states),  using  minimum  energy.  The  final-state 
constraints  can  be  written  mathematically  sls  Xd  —  e  <  x{tf)  < 
Xd  +  e. 

Our  objective  function  is  then  to  minimize  energy  consump- 
tion, while  satisfying  the  aforementioned  constraints.  Energy 
consumption  consists  of  two  parts  Jc  and  Jr,  corresponding 
to  capacitive  and  resistive  energy  losses  in  the  system,  respec- 
tively. The  piezoelectric  actuator  acts  as  a  capacitor  in  an  elec- 
tric circuit.  Hence,  the  major  part  of  the  energy  loss  happens 
when  actuators  are  charged  or  discharged.  This  corresponds  to 
a  transition  of  Uk  from  0  to  1  or  vice  versa.  This  energy  loss 
is  termed  as  capacitative  loss  or  .  In  a  general  on-off  con- 
trol case,  the  quantity  can  be  replaced  by  an  arbitrary 
"cost-to-switch" 

n 

JC=Y.  2^^max(K  -  Uk-lf  +  Ul)  (16) 

k  =  l 

where  C  is  the  capacitance  of  the  piezoelectric  actuator  and 
f^max  is  the  'on'  voltage  applied  to  the  actuators. 

The  second  part  of  the  objective  function  Jr  includes  energy 
lost  to  resistive  dissipation  due  to  leakage  current  in  the  on-off 
drive  circuit  or  through  the  piezoelectric  actuator,  and  is  given 
by 

"  TP 

Jr=J2 -f^TsUk  (17) 

k  =  0 

where  R  is  the  resistance  of  the  system.  In  other  words,  this  is 
the  energy  required  for  keeping  it^  at  1 .  Again,  here  the  quantity 
U^^^/Rcm  be  generalized  to  an  arbitrary  "cost-to-hold." 

The  optimization  problem  is  to  minimize  the  total  energy 
cost  J  =  Jc  -\-  Jr  subjected  to  state-dynamics  constraints  (n 
constraints),  binary  constraints,  and  final-state  constraints  given 
earlier.  Since  the  final  state  is  the  only  point  of  interest  here,  the 
state  dynamics  can  be  calculated  outside  the  optimization  (18) 
and  can  be  combined  to  the  final- state  constraints 

n-l 

x{tf)  =  x{nts)  =  Y,^d~'~'BdUi.  (18) 

i=0 


TABLE  III 

Nominal  Values  for  the  Parameters  Used  for  the  Design  of  Optimal 
Sequences  for  the  Systems 


Parameters 

Simulation 

Macro 

MEMS 

Model 

System 

Leg  Joint 

R  (Q) 

3*  10^ 

3*  10^ 

3  *  10" 

C(F) 

1  *  10-^ 

1  *  10-^ 

1  *  10-9 

UrnaxiV) 

20 

20 

20 

J  {kg.m2) 

1.4*10-12 

1 

1.4  X  10-^2 

b  (N.m.s/rad) 

3.4  *  10-11 

14 

2.7  X  10-^° 

k  (N.m/rad) 

3.2*  10-6 

40350 

4.4  X  IQ-^ 

G  {N.m/V) 

8*  10"^ 

64151 

2.8  X  10"^ 

Sampling  time  (Ts)(sec) 

0.0001 

0.001 

0.0001 

Now,  the  final-state  constraints  take  the  form 

n-l 

Xd-e<^  KiUi  <Xd-\-e  (19) 

i=0 

where 

The  total  energy  consumption  can  be  rewritten  as  follows: 

J  =  Jc+Jr 

k=l  k=0 

(20) 

Now,  this  problem  is  a  binary  programming  problem  with  a 
quadratic  objective  function  (20)  and  a  system  of  linear  con- 
straints (19).  Hence,  it  can  be  modeled  directly  in  dynamic 
programming  software  such  as  A  Mathematical  Programming 
Language  (AMPL)  and  solved  using  the  integer-programming 
solver  CPLEX,  which  uses  the  branch  and  bound  technique  to 
integer  programming. The  parameter  values  used  for  optimiza- 
tion are  given  in  Table  III. 

C.  Modification  to  Find  a  Robust  Sequence 

Uncertainty  in  the  inertia,  damping  and  stiffness  estimates 
(either  from  analytical  or  experimental  methods)  affects  the 
performance  of  the  system,  as  does  variation  in  actuator  behavior 
due  to  environmental  changes.  It  is  useful  to  minimize  the  error 
for  the  worst-case  system  with  a  bounded  uncertainty  Abound- 
This  is  a  minimax  problem,  which  can  be  represented  as  the 
following: 

min{max||a;i  ||A||  <  Abound-  (21) 

u  A 

Since  the  direct  solution  of  this  is  numerically  infeasible,  the 
algorithm  given  in  Fig.  9  and  described  in  the  following  was 
developed  to  find  an  input  sequence  that  gives  a  satisfactory 
performance. 

Two  optimization  techniques  are  employed  here.  Continuous 
nonlinear  optimization  for  determining  the  worst-case  systems 
(A  values)  and  binary  programming  for  finding  the  best  ro- 
bust input  sequence.  In  this  particular  problem,  three  param- 
eter uncertainties  are  considered,  namely  A^,  A^,  and  Aj^ 
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Fig.  9.    Robust  sequence  algorithm. 

corresponding  to  uncertainties  in  inertia,  damping,  and  stiff- 
ness. It  is  assumed  that  these  uncertainties  are  bounded  on 
either  sides.  But,  the  maximization  of  error  with  respect  to 
the  As  is  not  a  convex  optimization.  Hence,  the  optimization 
was  initialized  at  a  number  of  values  and  the  corresponding 
worst-case  system  for  each  was  found.  For  the  procedure,  let 
(Ai ,  ^1 ) ,  {A2 ,  ^2 ) ,  •  •  • ,  {Am ,  Bm )  be  the  dynamics  of  each  of 
the  worst-case  systems  obtained  by  the  maximization.  Then,  the 
final- state  constraints  for  all  m  systems  are  added  to  the  binary 
programming  to  find  the  robust  u  as  follows: 

n-l 

i  =  0 
n-l 

Xd-e<^  A^-'-^B2U^  <Xd  +  e 

i  =  0 


n-l 

Xd-e<Y^  A^'^'-^BmU,  <Xd^e.  (22) 

i  =  0 

These  constraint  equations  help  the  designer  to  identify  a 
specific  input  sequence  u  such  that  if  it  exists,  it  will  keep  the 
final  states  of  all  the  aforementioned  systems  within  some  e 
neighborhood  of  the  desired  final  state. 

V.  Simulation  Results 

JO  +  bd  +  kO  =  Gu.  (23) 

The  system  given  in  (23)  and  the  corresponding  parameters  are 
given  in  Table  III  was  used  for  initial  simulational  study.  A 
couple  of  sample  system  responses  using  the  minimal  energy 
open-loop  on-off  optimal  controller  is  given  in  Figs.  10  and  11. 
In  these  examples,  a  single  leg  link  is  driven  to  a  desired  final 
angle.  When  only  a  single  leg  link  is  to  be  controlled,  the  control 


Fig.  10.  Sample  simulated  system  output  using  optimal  on-off  controller  with 
loose  positioning  constraint,  showing  (a)  output  angle  and  (b)  system  input  u. 
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Fig.  11.  Sample  system  output  using  optimal  on-off  controller  with  strict 
positioning  constraint,  showing  (a)  output  angle  and  (b)  system  input  u. 

input  can  be  quite  simple,  as  in  the  example  shown.  In  Fig.  10, 
the  input  switches  twice  and  when  the  constraints  on  states 
are  stringent  the  controller  needed  three  switches,  as  shown  in 
Fig.  1 1 ,  which  corresponds  to  a  cheaper  controller  or  higher  state 
cost  as  in  linear-quadratic  Gaussian  (LQG)  controller  terminol- 
ogy. To  explore  controller  behavior,  this  on-off  controller  was 
compared  to  the  LQG  controller  for  the  system,  had  feedback 
and  analog  rather  than  on-off  inputs  been  available.  Two  LQG 
responses  corresponding  to  cheap  and  expensive  controllers  are 
given  in  Figs.  12  and  13.  Both  the  on/off  and  LQG  controllers 
produce  qualitatively  similar  trajectories  from  the  initial  to  the 
final  value.  In  addition,  the  capacitative  portion  of  the  cost  func- 
tions from  the  respective  controllers  is  found  to  be  less  for  the 
optimal  on-off  controller,  due  to  the  minimal  number  of  tran- 
sitions that  it  dictates.  The  PZT  actuators  used  in  the  prototype 
system  have  very  large  resistance,  such  that  over  99%  of  energy 
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Fig.  12.  Simulated  LQG  response  with  a  cheap  controller,  analogous  to 
strict  positioning  constraint,  showing  (a)  output  angle,  (b)  system  input  u,  and 
(c)  derivative  of  input. 


Fig.  13.  Simulated  LQG  response  with  an  expensive  controller,  analogous  to 
a  loose  positioning  constraint,  showing  (a)  output  angle,  (b)  system  input  u,  and 
(c)  derivative  of  input. 


use  is  due  to  capacitive  switching  losses,  as  opposed  to  resistive 
holding  losses. 

The  following  studies  are  done  to  make  sure  that  the  op- 
timization is  indeed  optimal  and  it  can  perform  effectively 
in  the  presence  of  uncertainty  in  the  system  parameters  or 
disturbances. 

A.  Comparison  to  Brute  Force  Methods 

To  ensure  that  the  optimization  method  is  working  properly, 
the  result  of  the  efficient  optimization  method  was  compared 
with  that  of  brute  force  optimization  for  a  shorter  time  period. 
In  the  brute  force  method,  all  possible  combinations  of  u[s 
were  checked  for  feasibihty  and  that  with  lowest  cost,  measured 
by  the  on-off  cost  function,  was  selected.  This  approach  is,  of 
course,  numerically  inefficient,  as  one  must  check  the  cost  for 


Fig.  14.  (a)  Response  of  the  perturbed  systems  when  a  robust  sequence  is 
applied;  the  red  lines  show  the  worst  case  systems,  (b)  Corresponding  robust 
on-off  sequence. 


2^  combinations  of  input,  making  it  impractical  for  a  larger 
time  period.  However,  it  is  possible  to  verify  that  for  our  sam- 
ple system,  the  result  of  the  brute  force  method  matches  the 
result  obtained  by  the  branch  and  bound  optimization  method, 
which  is  computationally  much  more  efficient.  Over  the  shorter 
time  period  analyzed,  the  optimal  control  sequence  identified  by 
integer  programming  exactly  matched  the  best  of  all  possible 
control  sequences  tested  through  the  brute  force  method. 

B.  Robustness  Analysis  Results 

From  the  simulations,  it  was  found  that  if  the  parameter  vari- 
ation (in  all  parameters)  is  within  10%  of  the  nominal  value 
for  the  sample  system,  then  it  was  possible  to  find  an  on-off 
sequence,  which  keeps  the  final  state  within  10%  of  the  desired 
final  state.  This  range  of  parameter  uncertainty  is  sufficient  to 
account  for  hysteresis  in  the  piezoelectric  response,  variation  in 
the  damping  coefficient  during  leg  motion,  and  potential  change 
in  piezoelectric  properties  over  an  approximately  10°C  range. 
Examples  of  perturbed  systems  are  shown  in  Fig.  14.  When  the 
parameter  variations  are  kept  within  ±10%  of  nominal  values, 
the  final  states  of  the  randomly  perturbed  system  are  within 
±10%  of  the  nominal  value,  0.3  ±  0.03,  and  this  was  consistent 
over  the  situations  we  examined. 

C.  Behavior  of  the  System  in  the  Presence  of  Disturbance 

A  natural  limitation  of  open-loop  control  is  its  inabihty  to  re- 
ject disturbances.  Additional  simulation  studies  were  performed 
to  explore  the  sensitivity  of  the  leg  joint  under  open-loop  on-off 
control  to  disturbances.  While  an  external  force  is  difficult  to 
apply  to  the  experimental  system  due  to  its  small  size,  simulated 
behavior  of  the  system  in  the  presence  of  various  disturbance, 
such  as  due  to  gravity  when  the  system  is  tilted,  friction  forces 
or  electrostatic  forces,  can  be  examined.  In  the  sample  simula- 
tion result  shown,  ideal  dynamic  friction  between  the  leg  and 
the  ground  was  considered  as  a  disturbance.  A  normal  force 
between  leg  and  ground  of  5  mN  and  coefficient  of  dynamic 
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Fig.  15.  Comparison  between  the  nominal  response  and  response  in  the  pres-  Fig- Hysteresis  curve  for  the  MEMS  actuator  with  the  dotted  lines  showing 
ence  of  ideal  friction.  the  variation  in  slope. 


friction  of  0.3  were  assumed  to  be  present.  The  deviation  in 
response  from  the  nominal  response  of  the  simulated  system  is 
shown  in  Fig.  15,  with  an  approximately  15%  reduction  in  dis- 
placement. As  this  normal  force  is  much  larger  than  the  payload 
capacity  of  the  leg  (5-50  mg,  corresponding  to  approximately 
0.5  mN)  and  much  larger  than  electrostatic  forces  operating 
over  the  displacement  range  of  the  leg,  there  is  reason  to  be- 
heve  that  disturbance  forces  that  would  not  cause  damage  to  the 
leg  joint  result  in  comparatively  small  errors.  This  is  primarily 
a  consequence  of  the  large  forces  for  this  scale  generated  by 
the  thin-film  piezoelectric  actuators.  On  the  other  hand,  because 
there  is  no  feedback  available,  the  system  has  hmited  capacity  to 
perform  a  disturbance  rejection  to  reduce  this  positioning  error. 
When  energy  constraints  are  so  strict  as  to  preclude  sensor  use, 
as  may  be  the  case  for  a  microrobot,  the  additional  positioning 
error  must  be  accepted.  However,  if  occasional  sensor  measure- 
ments can  be  made,  it  may  be  possible  to  adapt  the  input  on-off 
sequence  over  multiple  movements,  or  to  switch  between  opti- 
mized on-off  sequences  if  a  disturbance  is  detected,  in  order  to 
respond  to  disturbances  or  environmental  changes. 

VI.  Experimental  Results 

A.  Experimental  System  Description 

Experimental  testing  of  the  on-off  control  algorithm  was 
tested  on  both  a  macroscale  piezoelectric  test  actuator  and  the 
prototype  MEMS  leg  joint.  Before  conducting  the  experiments 
on  the  MEMS  actuator,  the  control  scheme  was  tested  on  a 
macroscale  piezoelectric  actuator  to  verify  the  controller  de- 
sign procedure.  The  macroscale  actuator  was  a  40-mm-long, 
10-mm-wide  Ceratec,  Inc.  bimorph  actuator  with  a  strain  gauge 
attached  to  it  for  measuring  the  deflection  in  terms  of  the  volt- 
age through  its  sensing  circuitry.  The  on-off  switching  sequence 
was  loaded  into  a  TMS320F28335  microprocessor,  which  was 
interfaced  to  the  bimorph  actuator  through  the  fast-switching 
circuit  explained  in  the  previous  section.  The  output  voltage 
was  measured  using  a  Tektronix  TDS2024B  oscilloscope  and 
data  was  captured  using  National  Instruments  Signal  Express 
Tektronix  Edition  Software.  From  the  step  response  of  the  sys- 


tem, the  following  second-order  system  given  was  identified 
between  input  voltage  and  strain  gauge  voltage: 

y{s)  64151 

=  .  (24) 

u{s)         +  14s  +  40350 

The  MEMS  actuator  was  also  operated  using  the  micropro- 
cessor and  the  switching  circuit.  Prototype  joint  arrays  were 
connected  to  the  switching  circuit  using  ultrasonic  wire  bond- 
ing to  bond  pads  at  the  base  of  the  actuator,  on  the  fixed  portion 
of  the  substrate.  The  switching  circuit  was  driven  by  the  micro- 
processor or  a  function  generator,  as  appropriate.  The  motion  of 
the  leg  was  captured  using  a  high-speed  camera  at  4000  frames 
per  second  and  the  angle  of  rotation  was  measured  using  the 
MATLAB  Image  Processing  Toolbox.  Using  the  step  response 
given  in  Fig.  7,  a  second-order  system  was  identified  between 
the  input  voltage  and  the  angle  of  rotation  (radians) 

y{s)  ^  1.9656  X  10^  

u{s)      s2  +  2  X  0.0547  x  1766.4s  +  1766.42  * 

To  measure  hysteresis,  a  static  input  voltage  was  varied  from 
0  to  20  V  and  back,  giving  the  hysteresis  plot  in  Fig.  16.  This 
hysteresis  effect  is  included  as  an  uncertainty  in  stiffness  for 
designing  a  robust  sequence,  which  is  used  in  the  experimental 
result  discussed  in  the  next  section. 

B.  Comparison  of  Experimental  and  Simulation  Results 

A  comparison  of  the  responses  from  the  macro  system  is 
shown  in  Fig.  17.  The  optimization  constraint  on  final  output  in 
this  example  was  to  reach  0.5  ±  0.1  V  in  strain  gage  output  at 
20  ms.  A  binary  optimization  was  done  on  the  identified  macro 
system  given  in  (24)  to  obtain  the  input  sequence  shown  in 
Fig.  17(b)  and  the  constraints  are  verified  using  the  MATLAB 
simulation  shown.  The  same  input  sequence  was  apphed  on 
the  macro  system  and  is  shown  in  Fig.  17(a).  The  experimental 
response  follows  very  closely  with  the  simulation  and  reaches 
0.58  V  at  the  desired  time. 

A  similar  approach  with  additional  robustness  constraints  to 
account  for  hysteresis  effects  was  apphed  to  the  MEMS  ac- 
tuator given  in  (25).  In  the  optimization,  the  constraints  were 
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Fig.  17.  Comparison  of  results  obtained  by  experiment  and  simulation  for  the 
macroscale  system  and  the  corresponding  switching  sequence. 
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Fig.  18.  (a)  Comparison  of  displacement  results  obtained  by  experiment  and 
simulation  for  the  MEMS  system  when  under  an  optimal  on-off  sequence 
for  0.15  rad  final  displacement,  (b)  On-off  voltage  input  applied,  (c)  Angular 
velocity  observed  in  the  simulation,  showing  successful  return  to  0  rad/s  at  final 
time. 


applied  on  both  angle  of  rotation  as  well  as  angular  velocity  at 
4  ms  for  all  the  perturbed  systems  with  stiffness  varying  be- 
tween A^min  =  127.906  V/rad  to  A^^ax  =  190.476  V/rad,  which 
is  about  ±20%  of  the  nominal  value  /^nominal  =  158.73  V/rad. 
The  requirement  was  to  make  the  angle  of  rotation  reach  0.15  ± 
0.03  rad  and  angular  velocity  reach  Oil  rad/s.  From  the  opti- 
mization, it  was  found  out  that  the  minimum  possible  tolerance 
on  angle  is  ±0.03  rad  because  of  the  20%  uncertainty  in  the 
stiffness  value.  The  input  sequence  and  simulation  responses 
shown  in  Fig.  18  were  obtained  as  a  result,  which  satisfies  all 
the  constraints  in  the  simulation.  When  the  same  sequence  was 
applied  on  the  physical  system,  the  response  shown  with  a  red 
line  was  obtained.  The  experimental  result  follows  the  simula- 
tion result  closely  and  reaches  about  0. 1 1 15  rad  at  the  stipulated 
time,  which  is  very  close  to  the  expectation  from  the  simula- 


Fig.  19.  Current  consumption  of  switching  circuitry  and  the  MEMS  actuator 
while  the  switch  is  (a)  turned  on  and  (b)  turned  off. 


tion  (misses  the  constraint  by  0.0085  rad).  Also,  the  direction 
of  rotation  reverses  in  the  same  video  frame,  indicating  that  the 
angular  velocity  goes  through  zero  verifying  the  final- velocity 
constraint.  The  discrepancies  between  the  experimental  and  de- 
sired motions  are  a  result  of  nonlinearity  of  the  actual  system, 
particularly  hysteresis  in  the  piezoelectric  film.  The  robust  de- 
sign approach  aids  in  ensuring  states  are  near  desired  final  values 
even  in  the  presence  of  model  error,  but  does  not  ensure  any  par- 
ticular accuracy  at  intermediate  times. 

During  this  experiment,  the  cumulative  energy  consumption 
of  the  microrobotic  leg  together  with  the  switching  circuitry 
was  also  measured.  The  current  profiles,  while  the  switch  was 
turned  "on"  and  "off  were  measured  using  a  current  probe 
and  oscilloscope  and  are  shown  in  Fig.  19.  The  power  supply 
was  kept  constant  at  20  V  during  the  entire  experiment  and 
each  turning  "on"  cost  4.6  x  10~^  J  and  each  turning  "off  con- 
sumed 1.4  X  10"'^  J.  Of  this  quantity,  2.2  x  10"'^  J  is  attributed 
to  charging  at  the  microactuator,  resulting  in  total  energy  loss  in 
the  circuit  alone  of  just  3.8  x  10~^  J  per  cycle  (the  difference 
between  total  "on"  and  "off  energy  usage  and  the  energy  re- 
quired to  charge  the  piezoelectric  capacitance).  This  is  smaller 
than  the  predicted  energy  usage,  which  appears  to  be  due  to 
additional  resistance  within  the  leg  between  electrodes  for  the 
leg  and  the  piezoelectric  actuators;  the  extra  resistance  further 
increases  the  effect  of  the  pull-up  resistors  included  in  the  circuit 
design.  Total  power  consumption  of  a  robot  using  this  controller 
would  depend  on  step  frequency,  but  for  walking  gates  of  20  Hz 
or  lower,  power  consumption  would  be  in  the  tens  of  microwatts 
or  smaller.  This  is  within  the  power  availability  we  predict  for  a 
microrobot  based  on  piezoelectric  actuators,  and  much  smaller 
than  power  consumption  of  an  analog  controller  or  PWM  con- 
troller with  much  higher  switching  frequencies. 

VII.  Discussion 

Although  the  optimal  control  method  discussed  earlier  can 
be  extended  to  include  feedback  by  using  a  model  predic- 
tive control  approach,  the  use  of  open-loop  on-off  control  to 
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regulate  the  motion  of  a  piezoelectric  actuator  is  driven  entirely 
by  the  need  to  ensure  extremely  low-power  consumption  from 
the  entirety  of  a  servo  control  system.  This  results  in  significant 
tradeoffs  in  performance  for  the  sake  of  power  reduction.  The 
simple  switching  interface  between  controller  and  actuators, 
and  small  number  of  transitions  utilized,  allows  for  control  of 
microactuators  that  act  primarily  as  a  capacitive  load  with  very 
little  energy  consumption,  less  than  a  microjoule  per  leg  motion. 
However,  such  a  controller  explicitly  forgoes  the  use  of  feed- 
back to  improve  response  time,  robustness,  or  other  closed-loop 
controller  benefits  due  to  power  limitations,  and  the  relatively 
low- switching  frequency  results  in  oscillatory  output  motions 
with  specific  desired  output  states  being  achieved  only  at  a  spe- 
cific time.  In  addition,  switching  controllers  may  often  excite 
high-frequency  dynamics,  though  this  is  not  a  large  effect  in  the 
experimental  test  actuators  examined  here.  If  present,  higher 
order  dynamics  may  be  incorporated  into  the  optimization  pro- 
cedure described  here  by  expanding  the  system  order,  but  the 
controller  will  only  act  to  ensure  that  these  dynamics  do  not 
influence  behavior  at  the  final  time,  not  to  avoid  vibration  or 
oscillation  at  intermediate  transitions. 

The  open-loop,  on-off  control  strategy  presented  here  func- 
tions best  when  the  system  model  is  well  known,  and  distur- 
bances or  uncertainty  in  modeling  have  known  and  bounded 
magnitudes.  For  example,  in  the  MEMS  application  discussed 
here,  analytical  modeling  of  the  piezoelectric  rotational  joint  be- 
ing controlled  is  useful  in  initial  actuator  design  and  setting  up 
controller  parameters.  It  may  be  used  to  identify  necessary  sam- 
pling periods  and  on-off  drive  circuit  response  times  as  well  as 
ranges  of  probable  system  parameters  (such  as  spring  stiffness  or 
damping).  However,  the  accuracy  of  analytical  models  is  limited 
by  variation  in  fabrication  processes  that  is  difficult  to  predict 
before  the  actuator  is  completed.  For  instance,  underetching 
of  the  silicon  layer  beneath  individual  actuators  and  residual 
stress  in  the  thin-film  piezoelectric  and  gold  layers  forming  the 
actuators  results  in  a  significant  reduction  in  joint  displacement 
from  its  ideal  performance.  In  addition,  uncertainty  in  exact  etch 
progress  and  light  upward  tilt  of  the  entire  joint  causes  viscous 
drag  on  the  leg  joints  to  vary  over  the  course  of  motion,  with 
resulting  nonlinear  deviations  in  damping  coefficient.  Once  a 
leg  joint  is  in  use,  an  experimentally  identified  model  is  best 
used  for  final  controller  design.  Even  so,  the  on-off  controller 
has  no  direct  ability  to  adjust  to  variations  in  the  plant  due  to 
environmental  changes  or  external  disturbances.  It  may  only  be 
designed,  as  described  in  this  paper,  to  minimize  error  over  a 
range  of  potential  variations  and  disturbances. 

Nonetheless,  on-off  switching  control  has  a  tremendous  ad- 
vantage in  terms  of  power  consumption,  as  switching  an  on-off 
signal  at  select  time  points  requires  much  less  power  than  ana- 
log drive  circuitry  or  high-frequency  PWM  inputs.  Because  of 
its  very  small  energy  usage,  this  controller  could  provide  useful 
baseline  control  for  autonomous  microrobotics.  On-off  control 
with  a  desired  final  time  is  especially  well  suited  to  quasi- static 
walking,  with  multiple  legs  driven  through  coordinated  motions 
over  a  specified  time,  at  which  legs  in  contact  with  the  ground 
are  raised  and  raised  legs  lowered  for  the  next  step,  and  absolute 
precision  of  an  individual  leg  is  not  as  important  as  assured  for- 


ward progress.  In  such  an  application,  total  inertia  of  the  system 
would  be  much  larger  than  that  of  the  leg  alone,  while  damping 
in  the  system  will  depend  on  the  geometry  of  the  body  and  feet 
as  well  as  legs.  In  practice,  the  environment  and  loads  on  the 
robot  may  change  dramatically,  requiring  some  level  of  adapta- 
tion or  feedback.  A  variety  of  optimized  on-off  sequences  under 
different  load  conditions  may  be  stored  for  use,  or  the  optimal 
on-off  sequence  may  serve  to  initiate  adaptive  control  using 
only  occasional  or  very  low  frequency  sensor  feedback  to  make 
modest  adjustments  to  the  input  sequence,  while  keeping  power 
consumption  low. 

VIII.  Conclusion 

The  chief  conclusions  of  this  paper,  then,  are  based  on  the 
idea  that  while  thin-film  piezoelectric  actuation  may  one  day  en- 
able unique  mobility  capabilities  from  microscale  autonomous 
robots,  achieving  directed  appendage  movements  will  require 
aggressive  reduction  in  power  consumption  throughout  the  servo 
control  system.  In  this  paper,  we  focus  on  modeling  leg  dy- 
namics and  reducing  power  consumption  by  the  actuators  and 
in  the  drive  circuit  interfacing  low- voltage  control  electronics 
with  a  high- voltage  actuator  supply.  In  particular,  we  introduce 
a  simple  optimization  method  for  achieving  minimum  energy 
on-off  control  when  switching  costs  are  substantial.  We  have 
applied  the  control  algorithm  to  both  a  macroscale  piezoelectric 
test  bed  and  to  prototype  microrobotic  leg  joints,  successfully 
directing  the  states  of  the  systems  to  desired  target  values.  In 
addition,  with  use  of  an  inverter  circuit  specifically  designed 
for  low-power  operation,  we  can  limit  leakage  current  while 
driving  the  actuators  and  obtain  our  desired  motions  with  just 
a  few  microjoules  per  leg  step,  within  the  predicted  energy  and 
power  limitations  of  future  microrobotic  platforms,  and  well 
below  more  conventional  control  implementations.  Controlled 
motions  are  completed  successfully  with  better  than  10%  posi- 
tioning accuracy  given  a  linear  model,  and  better  than  15%  in 
simulation  against  a  disturbance  forces  greater  than  1  mN,  and 
thus,  larger  than  we  would  expect  the  actuator  to  face.  How- 
ever, further  improvements  in  accuracy  are  difficult  given  the 
ultralow  power  open-loop  strategy,  such  that  further  robustness 
to  parameter  variation  or  disturbances  would  require  a  true  feed- 
back implementation,  and  thus,  increased  servo  system  power. 
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Abstract — ^As  a  novel  approach  to  future  microrobotic  locomo- 
tion, a  multi-degree-of-freedom  (m-DoF)  microrobotic  appendage 
is  presented  that  generates  large  range  of  motion  (5° -40°)  in  mul- 
tiple axes  using  thin-film  lead  zirconate  titanate  (PZT)  actuators. 
Due  to  the  high  driving  force  of  PZT  thin  films  and  a  robust 
fabrication  process,  m-DoF  legs  that  retain  acceptable  payload 
capacity  (~2  mg  per  leg)  are  achieved.  The  fabrication  process 
permits  thin-film  PZT  actuator  integration  with  more  complex 
higher  aspect  ratio  silicon  structures  than  previous  related  pro- 
cesses, using  vertical  silicon  dioxide  barrier  trenches  formed  be- 
fore PZT  deposition  to  provide  robust  encapsulation  of  the  silicon 
during  later  XeF2  release.  Planarization  of  the  barrier  trenches 
avoids  detrimental  effects  on  piezoelectric  performance  from  the 
substrate  alteration.  Once  fabricated,  kinematic  modeling  of  com- 
pact PZT  actuator  arrays  in  prototype  leg  joints  is  compared 
to  experimental  displacement  measurements,  demonstrating  that 
piezoelectric  actuator  and  assembled  robot  leg  joint  performance 
can  be  accurately  predicted  given  certain  knowledge  of  PZT  prop- 
erties and  residual  stress.  Resonant  frequencies,  associated  weight 
bearing,  and  power  consumption  are  also  obtained.  [2012-0041] 

Index  Terms — Actuators,  lead  zirconate  titanate  (PZT),  piezo- 
electric devices,  robots. 

1.  Introduction 

MICROROBOTS  are  a  frequently  discussed  potential 
application  of  the  ability  to  create  highly  engineered 
microdevices  synthesizing  microactuators,  sensors,  and  pro- 
cessing circuitry  [1].  Various  microactuation  mechanisms 
developed  for  walking  microrobotic  locomotion  have  been  re- 
ported in  the  literature,  but  existing  approaches  typically  feature 
limited  mobility,  large  power  requirements,  or  low  speed.  Thin- 
film  piezoelectric  actuation,  integrated  into  multi-degree-of- 
freedom  (m-DoF)  robot  legs  may  be  able  to  overcome  some 
of  these  limitations  on  autonomous  microrobots. 

Several  previous  researchers  have  demonstrated 
microelectromechanical- systems  (MEMS)-based  microrobots 
of  less  than  1  cm  in  length  using  various  actuation  mechanisms. 
For  example,  Ebefors  et  al.  built  a  silicon  walking  robot  with 
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single-degree-of-freedom  thermally  actuated  polyimide  leg 
joints  [2].  Although  the  polyimide  joint  structures  provided 
large  weight-bearing  capacity,  large  power  consumption  and 
low  speed  were  a  result  of  thermal  actuation.  This  tradeoff  was 
also  demonstrated  by  Murthy  et  al.  [3]  and  Mohebbi  et  al.  [7]. 
In  [4],  Hollar  et  al.  built  the  electrostatic  inch- worm  motors 
in  a  two-legged  microrobot.  However,  the  pin-joint  hinge 
mechanism  provided  a  single-degree-of-freedom  motion, 
although  power  consumption  was  very  small.  In  [5], 
Bergbreiter  and  Pister  integrated  electrostatic  inchworm 
motors  with  molded  elastomers  to  generate  a  rapid  jumping 
motion  from  a  similar  power  source  and  actuation  mechanism 
as  in  [4].  In  [6],  Donald  et  al.  used  electrostatic  scratch  drives 
that  were  actuated  by  electric  potential  difference  between  the 
actuator  and  the  substrate,  but  this  type  of  actuation  limits  the 
mobility  of  the  microrobot  to  a  certain  operating  environment. 

Piezoelectric  actuation  is  a  desirable  candidate  technology 
for  microrobotics  because  it  may  dramatically  increase  ap- 
pendage speed  and  has  large  force  capacity,  modest  voltage 
requirements  [16],  and  ability  to  recover  much  stored  electrical 
energy  using  charge  recovery  techniques  [19].  Recently,  lateral 
thin-film  lead  zirconate  titanate  (PZT)  actuators  were  developed 
in  [8]  that  demonstrated  very  large  work  densities  and  could  be 
connected  in  parallel  or  series  to  provide  large  force  capacities 
or  range  of  motion,  depending  on  application.  Lateral  thin- 
film  PZT  actuator  arrays  generating  series  rotation  between  5° 
and  20°  at  20  V  potentially  enable  comparatively  large  range 
of  motion  for  autonomous  microrobotic  appendages  [9],  [10], 
while  vertical  bending  piezoelectric  actuators  can  achieve  even 
larger  angles. 

A  first  topic  of  this  paper  is  to  introduce  basic  models 
of  out-of-plane  and  in-plane  piezoelectric  actuator  arrays  for 
microrobotic  leg  joints,  to  illustrate  the  critical  properties  of 
actuator  design.  The  strength  of  thin-film  PZT  actuation  for 
microrobotic  applications  is  the  combination  of  high  speed, 
relatively  large  angle,  and  low-power  operation  in  multiple 
axes,  as  compared  to  previous  technologies  in  Table  1.  The 
cost  for  this  improvement  on  the  various  prior  robotic  leg 
technologies  is  a  reduction  in  payload  capacity,  at  least  for  the 
type  of  leg  configurations  to  be  discussed  in  this  paper. 

The  second  topic  of  this  paper  is  a  robust  fabrication  pro- 
cess of  the  microrobotic  leg  joints  based  on  thin-film  PZT 
actuator  arrays.  Integrating  large  numbers  of  thin-film  piezo- 
electric actuators  into  high-performance  legs  with  multiple 
degrees  of  freedom  requires  high-yield  fabrication  of  both 
vertical  and  lateral  actuators  and  appropriate  connecting  struc- 
tures. Preferably,  associated  structures  should  be  composed  of 
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TABLE  I 

Comparison  of  Microrobotic  Devices 


Weight  Bearing 

Speed  [mm/s] 

DoF 

Maximum  Angle 

Volt 

Power 
Consumption 

Ebefors  (Thermal) 
Hollar  (Electrostatic) 
Murthy  (Thermal) 
Mohebbi  (Thermal) 
Current  Work 
(Piezoelectric) 

312.5  mg/leg 
5.1  mg/leg 
667  mg/leg 
1448  mg 

2.1  mg/leg 

6 

4 
1.55 
0.64 

27  (est.  max.) 

1 
1 
1 

3 

3 

-18.7" 
35" 
-2.9^ 
N.A. 

5",  40" 

18V 
50V 
lOV 
60V 

18V 

I.IW 
lOOnW 
750mW 
Not  specified 

60^W 

high-aspect-ratio  sihcon  or  other  hard  materials  to  maintain 
weight-bearing  capacity. 

Previous  processing  techniques  for  integrating  thin-film  PZT 
unimorph  benders  with  high- aspect-ratio  structures  have  lim- 
ited the  complexity  of  potential  microactuator  joint  arrays. 
Conway  and  Kim  demonstrated  an  SU-8  integrated  amplify- 
ing mechanism  in  [17].  Although  SU-8  could  be  chosen  for 
potential  microactuator  joint  arrays,  it  is  limited  to  single- 
level  processing  above  the  piezoelectric  film.  In  another  work, 
Aktakka  et  al.  developed  a  multilayer  PZT  stacking  process 
over  high-aspect-ratio  structures  in  [18].  Although  the  thickness 
of  a  thinning-stop  layer  could  be  chosen  for  various  target 
thicknesses  of  the  resulting  PZT  layer,  this  PZT  remains  thick 
compared  to  chemical- solution-deposited  thin-film  PZT  and 
thus  requires  large  voltages  for  equivalent  strains.  For  several 
energy-harvesting  devices,  such  as  that  in  [20],  silicon  is  left 
underneath  the  thin-film  PZT  layer;  this  can  be  acceptable 
for  energy  harvesting  from  vibration  at  such  devices'  natural 
frequency,  even  under  the  environmental  vibration  at  low  fre- 
quency. However,  timed  wet  etching  used  to  obtain  the  silicon 
cantilever  is  not  desirable  for  undercutting  the  PZT  layer  itself, 
which  limits  displacement  as  an  actuator. 

To  overcome  previous  processing  limitations  for  producing 
complex  microrobotic  joint  structures,  low-pressure  chemical 
vapor  deposition  (LPCVD)  of  silicon  oxide  vertical  barrier 
trenches  prior  to  thin-film  PZT  deposition  was  performed  in 
this  paper.  Compared  to  a  previous  photoresist  encapsulation 
technique  in  [11],  consistent  undercut  length  of  thin-film  PZT 
actuator  array  was  realized  that  is  essential  for  producing  the 
proposed  microrobotic  platforms  with  complex  joint  configu- 
rations. An  effective  piezoelectric  strain  coefficient  is  obtained 
with  minimal  change  in  magnitude  from  PZT  structures  without 
the  barrier  trenches. 

The  final  topic  of  this  paper  is  then  a  description  of  exper- 
imental testing  procedures  and  measured  robot  leg  behavior, 
as  related  to  range  of  motion,  response  speed,  and  power 
consumption.  After  fabrication,  a  prototype  leg  joint  is  charac- 
terized experimentally  to  validate  the  in-plane  and  out-of-plane 
joint  models.  Important  nonidealities  of  the  fabrication  process 
as  they  influence  robot  leg  joint  performance  and  appropriate 
adjustments  to  displacement  models  described  are  noted. 

To  summarize  paper  organization.  Section  II  introduces  the 
analytical  models  for  piezoelectric  vertical  and  lateral  actuators 
and  applies  them  to  the  actuator  arrays  produced  in  this  paper. 
Section  III  describes  the  fabrication  process  for  thin-film  piezo- 
electric layer,  integrated  with  complex  silicon  microrobotic 
structures.  Section  IV  presents  experimental  characterization  of 


fabrication  process  and  of  the  kinematics  and  dynamics  of  a 
sample  microrobotic  leg.  Section  V  concludes  this  paper. 

II.  System  Description 

A.  Thin-Film  PZT  Actuator  Basics 

Two  types  of  thin-film  piezoelectric  actuators  are  used  to 
create  M-DoF  appendages  such  as  that  shown  in  Fig.  1:  lateral 
(or  in-plane)  rotational  actuators  and  vertical  (or  out-of-plane) 
unimorph  bending  actuators  [15].  Individual  thin-film  PZT  ac- 
tuators consist  of  unimorph  bending  segments,  as  in  [8]:  a  single 
bend-down  uniform  segment  for  the  vertical  actuators  and  a 
combination  of  two  bend-down  and  two  bend-up  segments  for 
the  lateral  actuators.  Each  segment  contains  a  material  stack  of 
a  base  silicon  dioxide  layer,  a  bottom  Ti/Pt  electrode,  the  PZT 
thin  film,  and  a  top  Pt  electrode.  For  bend-down  segments,  an 
additional  gold  film  is  deposited  to  move  the  neutral  axis  of 
the  unimorph  above  the  centerline  of  the  PZT  film,  as  shown 
in  Fig.  2.  In  the  material  stack,  the  PZT  thin  film  both  imposes 
a  contractive  force  Fact  and  a  bend-down  or  bend-up  moment 
^act,i  or  Mact,2,  undcr  an  applied  voltage  according  to 

V 

^act  =e31,effT  ^PZT  (1) 

^PZT 

V 

^act,i  =  e31,eff   ^PZT(ypZT  "  Vi)  (2) 

where  esi^eff  is  the  effective  field-dependent  electroactive  stress 
coefficient,  which  is  a  measured  ratio  of  stress  to  electric  field 
rather  than  the  exact  linear  piezoelectric  coefficient  of  the 
material  [8].  The  coefficient  is  approximated  by  the  nominal 
(short  circuit)  axial  elastic  modulus  EpzT  of  the  PZT  film  in 
a  free  beam  and  by  the  effective  field-dependent  electroactive 
strain  coefficient  d^i^eS  or  631  ^eff  =  FpzT0^3i,eff-  The  empir- 
ical values  of  e3i^eff  and  (i3i,eff  include  a  number  of  effects 
associated  with  the  effective  piezoelectric  coefficient  0^31,/  or 
effective  piezoelectric  stress  coefficient  631  j  but  also  nonlinear 
piezoelectric/ferroelectric  and  electroactive  material  responses 
[11].  For  other  nomenclature,  V  is  the  applied  voltage,  tpzt 
and  Apzt  are  the  thickness  and  the  cross-sectional  area  of  PZT, 
ypzT  is  the  position  of  the  PZT  film  midline  in  the  unimorph 
material  stack,  and  yi  and  ^2  are  neutral  axes  of  segments  with 
(bend  down)  and  without  (bend  up)  a  gold  layer.  The  composite 
rigidity  of  respective  thin-film  PZT  unimorphs  is  referred  to 
as  (EI)  I  or  (EI) 2.  The  aforementioned  parameters  associated 
with  thin-film  PZT  actuator  structure  are  shown  in  Table  11. 
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-  Joint  a 


Before  Actuation 


After  Actuation 


Fig.  1.  Fully  released  m-DoF  leg.  (a)  Optical  image,  (b)  Schematic  drawing, 
(c)  Schematic  drawing  of  actuation  substructures  [(left)  before  actuation, 
(bottom  left)  before  actuation  of  PZT  film,  and  (bottom  right)  after  actuation 
of  PZT  film]. 


TABLE  II 

Properties  of  Thin-Film  PZT  Lateral  Rotational  and 
Vertical  Actuators 


Parameter 

Value 

Parameter 

Value 

Uu 

l(^m) 

^PZT 

480  (|um) 

tpt 

0.105  (fim) 

L, 

536  (^m) 

tpZT 

0.8  (|im) 

13  (^im) 

hi/Pt 

0.08  (^m) 

10  (^im) 

hi02 

0.17  (^im) 

1.693-102  (N•^m2) 

ypzT 

0.62  (^m) 

{EA\ 

11.20  (N) 

y\ 

1.05  (^m) 

[EI), 

6.909  (N-]um-) 

yi 

0.32  (^m) 

mi 

1.986  (N-]um2) 

llSC^m) 

F,,,  (at  20V) 

1.17-10-2(N) 

^tether 

284  (fim) 

^/,,,i(at20V) 

5.3-10-2  (N-^ni) 

^act 

15.5  (^im) 

Macta  (at  20V) 

3.7-10-2  (N-^im) 

B.  Thin-Film  PZT  Vertical  Actuator  Array 

Out-of-plane  motion  is  generated  by  simple  unimorph  bend- 
ing in  the  first  joint  (joint  a)  of  the  m-DoF  appendages  shown 
in  Fig.  1(a)  and  (b).  Displacements  of  any  actuator  array  can 
be  expressed  using  six  coordinates,  d  =  [x  y  z  Ox  Oy  Oz]^, 
although  only  specific  coordinates  of  significant  motion  are 
shown  in  each  respective  joint  analysis.  Since  all  out-of-plane 
or  vertical  actuators  are  composed  of  identical  silicon  dioxide, 
PZT  thin  film,  and  metal  layers,  a  multilayer  composite  can- 
tilever is  an  appropriate  model  to  estimate  the  displacement  of 


vertical  actuator 


^(a) 


(a) 
{EI)i  J 


Ma, 


+ 


3(£;/)i 


2{EI)^ 


L  2{EI)i        {EI)i  J 


r  1 

N 

L  N  J 

(3) 


Fig.  2.    Cross-sectional  view  of  a  lateral  rotational  actuator. 


where  Z(^a)  and  Ox^{a)  are  the  vertical  displacement  and  the 
rotational  angle  at  the  tip  of  joint  a,  Ly  is  the  length  of  a  vertical 
actuator,  Fy  and  My  are  external  force  and  moment,  if  present, 
due  to  the  external  loads  applied  at  the  tip  of  vertical  actuators, 
and  is  the  number  of  vertical  actuators  connected  in  parallel. 
A  schematic  view  of  this  motion  is  shown  at  the  bottom  of 
Fig.  1(c).  If  desired,  a  lateral  displacement  at  the  tip  of  the 
joint  ^(c^)  component  may  also  be  estimated  through  numerical 
integration  along  the  small  angle  displacement  of  O^^^a)  •  P^^- 
totype  leg  joints,  ten  vertical  actuators  are  connected  in  parallel 
such  that  large  weight-bearing  capability  can  be  maintained  by 
joint  a. 

As  imphed  by  (2)  and  (3),  vertical  actuator  joint  angles  can 
be  very  large  when  using  thin-film  PZT  unsupported  by  sili- 
con because  the  effective  piezoelectric  coefficient  is  relatively 
large,  electric  fields  are  high,  and  composite  stiffness  is  low. 
Downward  bending  motion  is  used,  with  the  added  gold  layer 
of  bend-down  unimorphs  both  increasing  offset  from  the  PZT 
layer  to  the  neutral  axis  and  preventing  composite  stiffness  of 
the  beams  from  becoming  too  low  in  resisting  external  loads. 

C.  Thin-Film  PZT  Lateral  Rotational  Actuators 

In-plane  motion  is  driven  by  individual  lateral  PZT  actua- 
tors [21],  [22]  integrated  with  silicon  flexural  structures.  The 
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Fig.  3.  Single  thin-film  PZT  lateral  rotational  actuator  connected  to  a  high- 
aspect-ratio  silicon  flexural  structure.  (Left)  Optical  image.  (Center)  Schematic 
drawing.  (Right)  Equivalent  spring  structure.  (Dashed-line  box)  Outline  of  the 
detailed  view  given  in  Fig.  6. 

net  motion  at  the  tip  of  individual  lateral  actuators  is  ideally 
horizontal,  as  two  bend-down  and  two  bend-up  segments  are 
located  symmetrically  on  either  side  of  the  actuator  midpoint. 
However,  due  to  residual  stress  in  the  thin  films,  unconstrained 
in-plane  motion  is  not  generally  obtained.  Thus,  the  end  of 
the  actuator  is  connected  to  a  high-aspect-ratio  silicon  flexural 
structure  that  prevents  out-of-plane  bending  under  most  circum- 
stances. The  flexure  structure  also  converts  small  translational 
piezoelectric  displacements  into  rotational  displacements,  as 
shown  in  Fig.  3. 

Force-displacement  curves  for  such  lateral  actuators  have 
been  previously  described  in  [8].  For  microrobotic  leg  design, 
a  region  of  small  displacement  and  large  force  is  chosen  to 
increase  the  weight-bearing  capacity  and  locomotion  speed  of 
joint  structures.  In  that  region  of  behavior,  the  axial  contraction 
in  (1)  dominates  actuator  motion,  while  the  bending  motion 
of  beams  that  can  lead  to  larger  displacements  against  small 
forces  has  negligible  influence  on  the  present  leg  joint  designs. 
Equation  (4)  shows  the  in-plane  small  displacement  of  a  single 
actuator  rotary  joint  that  consists  of  high-aspect-ratio  elastic 
flexure  and  tether,  a  PZT  actuator,  and  a  rigid  link.  The  relation 
of  the  force  generated  by  a  single  actuator  and  the  displacement 
at  the  tip  of  the  flexure  in  the  rotary  motion  is  as  follows: 

+  ^act,^)  0  =  F,,tL.,t  (4) 

where  kf^e^  h^e,  and  kg^ct^e  are  the  equivalent  rotational  spring 
stiffness  values  of  the  flexure,  tether,  and  actuator,  which  are 
given  in  the  Appendix.  Lact  is  the  distance  between  the  tether 
and  the  flexure  joint.  Because  the  actuator  itself  and  the  silicon 
tether  experience  the  axial  force  generated  by  piezoelectric  film, 
the  maximum  rotary  angle  0  is  represented  in  terms  of  the 
combined  spring  stiffness  of  the  joint  structure. 

D.  Thin-Film  PZT  Lateral  Actuator  Arrays 

For  intended  in-plane  actuation  in  microrobotic  legs,  multi- 
ple lateral  actuators  are  connected  in  series  arrays  so  that  large 
rotational  displacement  can  be  produced.  In  the  prototype  legs, 
this  occurs  in  joints  (3  and  7.  In  these  joints,  lateral  actuators 
are  arranged  in  two  sets  of  four  with  flexural  mechanisms 
facing  each  other,  as  shown  in  Fig.  1(b).  A  schematic  view 
of  this  motion  is  shown  in  Fig.  1(c).  By  aligning  the  high- 
aspect-ratio  flexures  toward  a  microrobot's  foot,  such  arrays  are 
intended  to  reduce  out-of-plane  deflection  due  to  torsion  from 


weight  bearing  of  future  microrobotic  platforms  and  to  multiply 
rotational  motion  of  individual  actuators. 

Within  each  in-plane  joint  array,  displacement  is  calculated 
from  the  driving  thin-film  PZT  actuation  force  and  its  equiva- 
lent rotary  moment.  Eight  local  coordinates,  as  shown  in  Fig.  4, 
are  used  to  calculate  the  total  displacement.  The  displacement 
of  the  origin  of  the  ith  coordinate  with  respect  to  the  first 
coordinate  at  the  start  of  the  array  (point  ^0  or  70)  is  represented 
as  the  summation  of  projections  onto  previous  coordinates. 
Then,  displacement  of  the  mth  coordinate  with  respect  to  the 
origin  of  the  first  flexure  joint  is  represented  in 

m  n 

^m(/3,7)=X]  n        (^,7)  (^''+^-/(''^,7)+^n+l(/3,7)^''^^) 
n=l k=l 

(5) 

where  d\  is  the  total  local  displacement  of  the  compact  ar- 
ray in  question,  is  the  rotational  matrix  of  the  ith  coordinate 
with  respect  to  jth  coordinate  by  the  angle  of  Oi,  and,  for  i  >  5, 
Rf  is  a  combined  transformation  of  the  reorientation  of  axis  by 
TT  radians  R{7t)  with  rotational  displacement  by  6iR{0i).  When 
1  =  1,  Ri  is  the  identity  matrix.  T  is  the  compliance  matrix 
of  the  flexure  structure,  p  is  the  forcing  term  generated  by 
the  actuator  and,  if  present,  external  forces  and  moments, 
is  the  initial  position  of  flexure  tip,  and  is  the  position  vector 
between  the  zth  and  i  —  1th  coordinates.  The  subscript  /3  or  7 
indicates  whether  the  model  is  referring  to  the  inner  or  outer 
compact  array. 

For  in-plane  actuator  design,  performance  is  best  if  the 
elastic  flexure,  denoted  by  kf^e^  can  be  fabricated  with  high 
aspect  ratio,  so  that  out-of-plane  stiffness  and  weight  bearing 
are  increased  without  overly  increasing  resistance  to  in-plane 
motion.  Likewise,  small  gaps  between  flexures  and  tethers  Lact 
are  also  useful  for  generating  large  rotation  angles.  Significant 
forcing  is  again  available  through  large  piezoelectric  coefficient 
and  high  electric  field  across  the  PZT  thin  film. 

E.  Kinematic  Leg  Model 

In  the  prototype  leg  configuration,  coordinate  systems  are  de- 
fined at  the  silicon  flexures  of  each  individual  in-plane  actuator, 
in  addition  to  the  initial  vertical  actuator.  As  shown  in  Fig.  1, 
the  following  locations  are  used  as  reference  frame  of  the  local 
coordinates  during  kinematic  analysis. 

1)  Point  0  represents  the  beginning  of  leg,  at  the  fixed  end  of 
the  vertical  actuators,  oriented  along  the  long  axis  of  the 
vertical  actuators. 

2)  Point  A  represents  the  tip  of  the  vertical  actuators  (end 
of  joint  a),  where  a  rigid  silicon  connection  to  joint  (3 
begins. 

3)  Points  /3o  and  B  represent  the  beginning  and  end  of  joint 
^,  at  the  end  of  the  rigid  sihcon  connector  between  joints 
a  and  (3. 

4)  Points  7o  and  C  represent  the  beginning  and  end,  respec- 
tively, of  joint  7,  at  the  end  of  the  rigid  silicon  connector 
between  joints  (3  and  7. 

5)  Point  D  is  the  end  tip  of  the  m-DoF  leg. 
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Fig.  4.    Schematics  of  local  coordinates  of  inner  and  outer  actuator  array  in  m-DoF  robotic  leg. 


6)  Point  E  is  the  corner  of  the  rigid  sihcon  hnk  between 
joints  P  and  7,  which  is  easily  tracked  during  experimen- 
tal measurements. 

The  local  coordinate  axes  for  each  joint  is  chosen  such  that 
the  2/0 -axis  is  aligned  along  the  lateral  direction  of  the  thin- 
film  PZT  actuators  in  the  given  joint.  Local  displacements 
are  related  to  global  displacements  of  points  of  interest  using 
rotation  matrices.  Fixed  rotation  matrices  are  denoted  by  the 
notation  i?(6>i  5/^),  where  i  corresponds  to  the  local  axis  of 
rotation  and  b  and  a  indicate  the  ending  and  starting  points  of 
interest,  respectively,  that  a  change  in  orientation  is  taken  from. 
As  an  example,  for  the  first  rotation  matrix,  from  the  global 
coordinates  to  the  base  of  the  vertical  actuators,  the  rotational 
matrix  is  formed  from  Oz^a/o  which  is  the  angle  between  global 
coordinates  (xgiobab  2/giobab  global)  and  the  local  coordinates 
{xo,yo,zo)  at  point  0,  the  beginning  of  the  leg.  From  the 
measured  distance  of  point  A  from  zero,  Oz^a/o  is  estimated 
at  39.5°,  and  coordinate  transformation  from  the  raw  data  to  the 
desired  coordinate  (xq,  ^o,  ^0)  is  obtained  in 


global 


R{0 


z,A/0 


(6) 


where  Xgiobai  and  Xq  are  the  position  vectors  with  respect  to 
the  global  and  point  0  coordinates. 

Rotations  that  may  change  with  apphed  voltage  are  also 
written  using  standard  rotation  matrices,  denoted  by  77(6^^  5/^), 
and  generally,  their  rotation  angles  consist  of  a  fixed  compo- 
nent, due  to  residual  stress,  and  a  changing  component  due 
to  applied  voltage.  For  example,  the  rotation  matrix  used  to 
transform  coordinates  about  point  A  with  respect  to  coordinates 
about  point  0  is  written  H{0^a/o)^  with  O^^a/q  being  the  sum 
of  vertical  rotations  at  the  tip  joint  a  due  to  residual  stress 
and  due  to  an  apphed  input  voltage.  Use  of  these  rotations  to 
evaluate  leg  joint  performance  will  be  discussed  in  more  detail 
in  Section  IV. 

III.  Fabrication 

A.  Robust  Encapsulation  and  Multilevel  Eabrication 
Integrated  With  Thin-Eilm  PZT  Layer 

A  previous  microfabrication  process  for  components  of  thin- 
film  PZT- actuated  microrobotic  leg  joints  has  been  reported  in 
[8].  Individual  actuators  were  released  from  underlying  silicon 


with  a  timed  XeF2  etch  when  a  photoresist  encapsulation 
layer  protects  silicon  anchor  points.  However,  this  resulted  in 
obstacles  to  releasing  the  undercut  PZT  actuator  layer  across 
large  devices,  such  as  inconsistent  actuator  undercut  length  and 
occasional  crack  and  failure  of  photoresist  encapsulation  layer. 

A  robust  fabrication  process  used  to  produce  the  current 
m-DoF  appendages  is  as  follows.  Fabrication  begins  with  a 
silicon-on-insulator  (SOI)  wafer  with  a  lO-ytxm-thick  device 
layer,  O.l-^t/m- thick  buried  oxide  layer,  and  500-//m-thick  han- 
dle wafer  (thicker  device  layers  may  also  be  used,  with  legs 
from  30-//m  device  layers  also  fabricated).  Narrow  (3  fim  wide) 
trenches  are  etched  to  the  buried  silicon  dioxide  layer  (step  A) 
in  Fig.  5)  along  the  sidewalls  of  bulk-micromachined  silicon 
structures,  such  as  flexural  springs  and  sihcon  connectors  from 
the  tethers  to  PZT  unimorphs.  Prefurnace-cleaned  trenches  are 
then  filled  with  silicon  dioxide  by  LPCVD  [step  B)]. 

It  was  observed  that  keyholes  are  generated  in  the  oxide 
trenches  due  to  nonuniformity  in  the  trench  sidewalls  and  a 
shghtly  faster  deposition  rate  at  the  trench  mouth  than  that 
inside  the  trench.  Instead  of  annealing  above  1000  °C,  which 
has  been  shown  to  make  the  oxide  reflow  in  the  trench  [12],  the 
top  oxide  is  removed  by  either  chemical  mechanical  polishing 
or  reactive  ion  etching  (RIE),  and  the  upper  part  of  the  trench 
is  etched  by  short  RIE  [steps  C)  and  D)].  Once  the  keyhole  is 
opened,  LPCVD  process  is  repeated  to  fill  the  opened  trenches 
in  step  E).  Several  iterations  of  steps  B)-D)  may  be  necessary 
to  diminish  the  size  of  keyhole,  but  it  is  difficult  to  eliminate 
it  entirely.  Nonetheless,  since  the  observed  piezoelectric  coeffi- 
cients of  PZT  films  can  be  very  sensitive  to  underlying  surface 
conditions  [13],  refill  is  continued  until  a  standard  mechanical 
boundary  condition  of  the  intermediate  layer  between  PZT  film 
and  the  substrate  is  achieved.  In  this  case,  closure  of  all  surface 
trenches  and  a  planarized  surface  are  achieved.  Planarization  of 
the  surface  of  refilled  oxide  trenches  is  followed  by  additional 
silicon  dioxide  deposition  by  plasma-enhanced  chemical  vapor 
deposition  to  reach  a  minimum  target  base  oxide  thickness  of 
2000  A. 

After  SOI  wafers  are  processed  up  to  step  E),  PZT  (with 
a  Zr/Ti  ratio  of  52/48)  and  electrode  layers  are  deposited  at 
the  U.S.  Army  Research  Laboratory  and  Radiant  Technologies, 
Inc.  The  resulting  metal  stack  consists  of  a  platinum  bottom 
electrode,  PZT  film,  and  platinum  top  electrode  [step  F)].  Two 
argon  milling  steps  for  the  top  electrode  and  PZT  layers  and 
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Fig.  5.    Multilevel  microfabrication  process  flow  of  thin-film  PZT-actuated  silicon  structure  in  the  microrobotic  application. 

a  wet  etch  for  contact  vias  to  the  bottom  electrode  are  next 
performed  [step  G)],  as  in  [8].  Then,  a  l-/im-thick  Ti/Au  layer 
is  deposited  on  the  top  of  PZT  actuator  surface  by  a  lift-off 
process  [step  H)],  and  the  silicon  microstructure  in  the  top 
device  layer  is  patterned  and  micromachined  by  deep  RIE 
(DRIE)  [step  I)]. 

To  perform  backside  etching  of  the  SOI  wafer,  oven-baked 
photoresist  is  used  to  protect  the  microstructure  of  top  device 
layer,  and  the  backside  of  SOI  wafer  is  patterned  for  other 
geometries  such  as  microrobot  feet.  Backside  patterning  and 
etching  also  provide  the  open  area  for  XeF2  to  etch  the  un- 
derneath silicon  layer  of  actuator  and  other  unprotected  silicon 
structures  in  step  J).  During  the  XeF2  underetching  process 
of  thin-film  PZT  actuator  and  silicon  microstructure  layer,  a 
uniform  undercut  length  of  the  PZT/Au  unimorph  actuator 
structure  is  obtained  after  removal  of  chemically  inert  Teflon- 
like passivation  layer,  which  is  deposited  during  DRIE  process 
and  prevents  XeF2  gas  from  etching  the  silicon  underneath  the 
material  stack.  Fig.  6  shows  a  scanning  electron  microscopic 
image  of  thin-film  PZT  actuator  layer  connected  to  silicon 
tether. 

To  summarize,  in  the  microfabrication  process  described 
earlier,  deposition  of  oxide  trenches  provides  a  vertical  etching 
barrier  against  isotropic  XeF2  etching  of  the  substrate  silicon 
device  layer  connected  to  thin-film  PZT  actuators.  This  serves 
as  a  robust  encapsulation  structure  to  eliminate  encapsulation 
failures  and  underetching  variance  of  thin-film  PZT  actuators 
observed  in  previous  integration  approaches.  Oxide  deposited 
for  such  trenches  can  also  be  used  as  a  hard  mask  on  the 
backside  of  a  SOI  wafer  permitting  backside  patterning  and 
thus  multilevel  fabrication  for  complex  silicon  structure.  Silicon 
structures  up  to  30  /jm  tall  by  5  /jm  wide  have  been  produced 
in  this  manner  on  other  test  wafers. 
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Fig.  6.  Scanning  electron  microscopic  image  of  thin-film  PZT  actuator  layers 
connected  to  silicon  tether,  as  marked  in  a  dashed  red  box  in  Fig.  2. 


One  limitation  of  the  finalized  process  flow  is  an  inability 
to  remove  the  oxide  trenches  following  XeF2  release.  While 
the  oxide  trenches  were  originally  intended  to  be  removed  via 
isotropic  RIE,  and  structures  were  designed  with  the  removal  of 
silicon  dioxide  expected,  in  practice,  the  top  platinum  electrode 
on  the  PZT  layer  was  found  to  be  eroded  in  such  an  RIE 
step,  causing  the  removal  of  electrical  connection  in  most  of 
actuators.  Thus,  flexible  structures  in  the  device  layer  are  wider 
than  originally  designed  by  the  oxide  trench  width,  and  flexible 
electrical  interconnects  (such  as  that  providing  power  to  the 
outer  in-plane  actuator)  are  also  reinforced.  This  reduces  the 
range  of  motion  of  the  final  leg  structures  compared  to  their 
designed  specifications,  and  future  devices  should  take  oxide 
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trench  width  into  account  in  the  design  stage.  In  addition, 
residual  stresses  in  the  metal  stack  result  in  the  vertical  actuator 
array  having  a  nonzero  neutral  position.  The  change  of  the 
neutral  position  leads  to  variation  in  vertical  displacements 
from  planar  neutral  position  which  would  be  expected  from  an 
initially  flat  structure. 


IV.  Experimental  Results 
A.  Piezoelectric  Coefficient  Measurement 

Following  fabrication,  simple  cantilever  beams  were  used  to 
verify  piezoelectric  coefficient  magnitudes.  This  was  done  to 
verify  that  the  preprocessing  of  the  SOI  with  barrier  trenches 
would  not  have  a  detrimental  effect  on  thin-film  PZT  quality. 
Since  a  randomly  oriented  polycrystalline  structure  is  present 
in  PZT  thin  film  after  fabrication,  a  poling  treatment  is  required 
to  align  the  scattered  polydomain  structure  into  a  metastable 
ahgnment  in  the  direction  of  an  applied  electric  field  to  form 
a  consistent  remnant  polarization  of  the  layer.  The  poling 
test  was  performed  at  250  kV/cm  (i.e.,  20  V)  using  a  direct- 
current  power  supply  at  room  temperature  which  was  used  for 
45  min.  After  poling,  the  static  vertical  displacement  of  the  tip 
of  cantilever  was  measured  by  optical  profilometry  by  stepping 
down  to  0  V  in  1-V  decrements  and  returning  back  to  20  V  in 
1-V  increments. 

Due  to  intrinsic  residual  thin-film  stress  following  fabrica- 
tion, the  piezoelectric  coefficient  is  experimentally  determined 
from  radius  of  curvature  of  the  beam  p  in  the  Bernoulli-Euler 
equation  with  the  superposition  of  the  resultant  moment  due 
to  the  residual  stress  and  the  moment  generated  by  PZT 
thin  film 
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Fig.  7.  Empirical  effective  electroactive  piezoelectric  strain  coefficient  curve 
at  applied  voltages. 
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Fig.  8.  Trajectory  of  point  E  (knee)  between  joints  /3  and  7  and  point  D  (foot) 
as  the  vertical  actuators  are  activated  by  15-V  dc.  The  left  images  are  at  0  V, 
and  the  right  images  are  at  15  V. 
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where  Mresiduai  is  the  moment  due  to  residual  stress  of  the 
films,  MpzT  is  the  moment  generated  by  PZT  actuator,  which 
corresponds  to  Mact,2  in  (2),  and  (^/)comp,no  Au  is  the  vertical 
composite  rigidity  of  the  material  stack  without  Au  deposition. 
Fig.  7  shows  the  estimated  field-dependent  effective  electroac- 
tive strain  coefficient  (iai^efr  of  the  piezoelectric  cantilevers 
with  and  without  oxide  barrier  trenches.  Multiple  measure- 
ments have  been  performed  at  each  applied  electric  field  and 
represented  as  error  bars,  and  the  electroactive  —  (i3i,efr  values 
for  both  test  cantilevers  at  higher  voltages  are  determined  in  the 
range  of  60-80  pm/V.  The  variation  of  measured  (iai^eff  with 
respect  to  electric  field  is  observed  inversely  proportional  to  the 
applied  electric  field  as  deflection  due  to  intrinsic  residual  stress 
is  predominant  at  low  voltage  and  the  piezoelectric  coefficient 
is  inversely  proportional  to  electric  field.  The  empirical  (isi^eff 
coefficients  in  Fig.  7  show  that  the  performance  of  fabricated 
actuators  could  be  preserved  in  the  proposed  microfabrication 
process,  particularly  at  high  voltages  where  typical  operation 
occurs. 


B.  Range-of-Motion  Analysis 

Completed  m-DoF  leg  joints  were  characterized  experimen- 
tally to  verify  functionality  and  vertical  and  lateral  joint  design 
performance.  This  experimental  verification  is  based  on  kine- 
matic descriptions  of  joint  motion,  with  deformation  of  contin- 
uous beams  (vertical  unimorph  actuators  and  elastic  rotational 
flexures,  for  out-of-plane  and  in-plane  motions,  respectively) 
converted  to  motion  of  specified  points  on  a  prototype  actuator 
in  global  coordinates  [14].  This  motion  was  captured  by  a 
high-speed  camera  above  the  device,  and  the  global  coordinate 
(a^^giobai,  2/giobai)  data  are  obtained  from  the  optical  images. 

As  discussed  in  (6)  in  Section  II-E,  displacements  of  any 
point  of  interest,  as  measured  in  global  coordinates,  can  like- 
wise be  related  to  rotations  and  displacements  of  proceeding 
links  in  the  robotic  leg.  For  example,  the  displacement  of  the 
tip  in  terms  of  global  coordinates  Xj:,  depends  on  all  relevant 
rotations  and  displacements,  as 

rans 

+  H{Ox^A/o)R  (^2,/3o/a)  '^trans  +  ^(^a;,A/o)'^trans 
+  '^trans  (9) 
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Fig.  9.  Trajectory  of  two  in-plane  compact  arrays  with  points  D  (foot)  and 
E  (knee)  represented  with  hollow  markers,  as  designed,  and  represented  with 
solid  markers,  as  measured,  with  respect  to  point  /3o  when  joint  a  is  activated. 


Fig.  10.  Trajectory  of  two  in-plane  compact  arrays  with  points  D  (foot)  and 
E  (knee)  represented  with  hollow  markers,  as  designed,  when  joints  /3  and  7 
are  activated  separately,  and  represented  with  solid  markers,  as  measured,  with 
respect  to  point  /3o  when  joint  /3  is  activated. 


where  H{0^^a/o),  ^(<9x,b//3o),  and  i^(<9^,c/7o)  are  the  local 
coordinate  transformations  due  to  rotations  at  joints  a,  /3,  and 
7,  respectively,  and  R{0z^/3o/a)  is  a  fixed  rotation  representing 
the  difference  in  orientation  between  joint  a  and  joint  /3.  Xj^  jc 
is  fixed  translational  offset  of  the  tip  of  the  leg  from  the  end 
of  joint  7,  5trans  IS  the  nominal  offset  from  the  beginning  of 
joint  7  to  the  end  of  joint  7  plus  any  displacement  of  joint  7 
(i.e.,  (ig  ^^-^  due  to  an  apphed  voltage,  i(;trans  is  the  nominal 
offset  from  the  beginning  of  joint  ^  to  the  end  of  joint  7  plus 
any  displacement  exerted  by  joint  ^  (i.e.,  d\  due  to  applied 
voltage,  ^trans  is  the  nominal  offset  of  joint  from  joint  a, 
and  i;trans  is  the  nominal  offset  from  the  beginning  to  the  end 
of  joint  a  plus  any  displacement  of  joint  a  due  to  an  applied 
voltage. 

Fig.  8  shows  the  top  view  of  the  device  and  the  trajectory 
of  E  (knee)  and  point  D  (foot)  taken  by  stereoscope  and 
high-speed  camera  when  the  vertical  actuators  are  activated  by 
various  input  voltages.  In  Fig.  9,  the  direction  of  motion  of 
points  D  and  E  exerted  by  joint  a  is  opposite  to  the  intended 
direction.  This  is  because  residual  stress  causes  the  leg  to  start 
in  an  upward  deflected  position,  such  that  downward  rotation 
of  the  vertical  actuator  under  an  applied  voltage  causes  the 
tip  of  the  leg  to  move  away  from  the  base  (in  global  {x,y) 
coordinates),  rather  than  closer  to  the  base  if  the  leg  had  started 
in  a  flat  fully  extended  position.  Furthermore,  residual  stress 
leading  to  the  deformation  of  joint  ^  causes  significant  out-of- 
plane  bending. 

Fig.  10  shows  the  measured  trajectories  of  points  D  and 
E  under  various  combinations  of  applied  voltage  to  joints  ^ 
and  7.  Differences  between  the  designed  and  the  measured 
trajectories  of  the  leg  also  arise  from  nonidealities  in  the 
fabrication  process,  particularly  the  following:  1)  excess  silicon 
dioxide  that  increases  the  stiffness  of  flexures  and,  particularly. 


TABLE  III 

Displacement  of  Point  A  by  Vertical  Actuators 


Degree  of 
Freedom 

As  designed 

As  fabricated 
(from  model) 

As  fabricated 
(from  experiment) 

x(^im) 

0 

0 

1.86 

y  (Mm) 

-43 

10.6 

19.4 

z(^un) 

-189 

-189 

-148 

-40 

-40 

-37 

0 

0 

0 

%.n 

0 

0 

0 

the  interconnect  across  the  inner  lateral  actuator  and  2)  residual 
stress  in  the  actuator  arrays. 

C.  Comparison  to  Actuator  Models 

Mapping  back  observed  motion  at  the  points  of  interest 
along  the  leg  to  displacements  at  the  actuator  arrays  allows  the 
actuator  array  performance  to  be  measured.  Since  the  motion 
of  m-DoF  microrobotic  legs  is  generated  by  combinations  of 
two  in-plane  compact  actuators  and  vertical  actuators,  differ- 
ent sets  of  actuators  were  independently  activated  to  verify 
actuator  models.  Three  sets  of  displacements  are  shown  in 
Tables  III-V,  for  the  three  actuators.  The  first  set  of  results 
shows  the  displacement  of  an  ideal  leg  design,  which  assumes 
a  planar  neutral  position  (no  residual  stress),  no  excess  silicon 
dioxide,  and  negligible  interconnect  stiffness  across  the  inner 
lateral  actuator;  this  set  of  results  is  intended  to  demonstrate  the 
ideal  capabilities  of  the  device.  The  second  set  of  results  shows 
the  predicted  displacements  when  residual  stress  effects  are 
included,  with  extra  oxide  and  with  full  interconnect  modeling. 
The  third  set  of  results  is  from  the  experimentally  extracted 
actuator  array  displacements  and  rotations,  when  measurable. 
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TABLE  IV 

Displacement  of  Point  E  With  Respect  to  Point  A  by  Inner 
Compact  Actuators 


Degree  of 
Freedom 

As  designed 

As  fabricated 
(from  model) 

As  fabricated 
(from  experiment) 

x(|um) 

230 

4.9 

3.8 

y(|nm) 

65 

-133 

-143 

z(^m) 

0 

843 

798 

0 

18.9 

18.1 

0 

0 

Unable  to  measure 

-4.7 

-0.1 

-0.1 

TABLE  V 

Displacement  of  Point  C  With  Respect  to  Point  E  by  Outer 
Compact  Actuators 


Degree  of 
Freedom 

As  designed 

As  fabricated 
(from  model) 

As  fabricated 
(from  experiment) 

x(^m) 

149 

144 

114 

y  (Mm) 

-208 

-201 

-186 

0 

0 

108 

0 

0 

17.7 

0 

0 

Unable  to  measure 

-4.7 

-4.5 

-3.85 

Based  on  experimental  observations  of  fabrication  limita- 
tions, various  adjustments  were  made  to  the  actuator  models. 
First,  it  should  be  noted  that,  even  with  high-aspect-ratio  flexure 
and  tether  structures,  residual  stress  in  the  experimentally  re- 
leased prototype  devices  described  in  Section  III  still  led  to  out- 
of -plane  bending  from  the  in-plane  actuators.  Thus,  an  addition 
to  the  lateral  actuator  model  to  account  for  out-of-plane  defor- 
mation was  obtained.  This  was  particularly  significant  when 
the  elastic  interconnect  is  present,  as  a  deformation  similar  to 
buckling  is  observed. 

To  account  for  the  stiffness  of  the  PZT  interconnect  to  joint 
7  across  joint  /3,  which  impeded  lateral  rotation  in  joint  ^  and 
contributed  to  out-of-plane  bending  moment  about  the  x-axis, 
an  expanded  model  of  the  lateral  actuator  array  with  intercon- 
nect was  created.  Details  of  the  augmented  model  are  included 
in  the  Appendix,  but  in  brief,  this  effect  can  be  estimated  by  the 
average  offset  distance  ^dist  of  the  beginning  of  the  rigid  silicon 
links  between  joints  (3  and  7  along  the  local  ^-direction  with 
respect  to  point  A  during  the  motion.  This  offset  distance  and 
in-plane  actuation  moment  result  in  out-of-plane  deflection  and, 
thus,  the  resulting  angle  Oic^y  of  the  thin-film  PZT  interconnect 
about  point  A.  A  lumped  spring  model  is  used  to  estimate  the 
displacement  of  the  parallel  structure  of  the  elastic  interconnect 
and  joint  /3,  which  is  represented  by  the  displacement  of  joint 

{(3) 

^8(/3)  ^  (-^ic,eq^ic  H"  -^array,eq)     -^act  (10) 

where  Kic,eq  and  /^array,eq  ^rc  the  lumped  spring  stiffness 
values  of  the  thin-film  interconnect  and  inner  in-plane  actuator 
array  and  Gic  is  a  matrix  projecting  the  additional  spring  force 
at  the  tip  of  the  interconnect  back  to  actuator  locations. 

In  general,  there  is  a  good  agreement  between  experimental 
and  model  results  when  nonidealities  are  taken  into  account. 
Angular  rotation  of  the  vertical  actuator  tip  is  closely  matched 
to  the  model,  although  the  initial  curvature  due  to  residual  stress 
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Frequency  (logift  Hz) 


Fig.  1 1 .  Frequency  response  of  point  C  (foot)  measured  with  respect  to  global 
coordinates  when  the  vertical  actuator  array  is  activated  with  driving  sinusoidal 
input. 


TABLE 

VI 

Resonant  Frequencies 

Actuator 

Measurement 

1^*  and  higher  resonant  frequency 

Vertical  actuator 

Point  A  (hip) 

300,  1500,  1810  Hz 

Point  B  (knee) 

37,  300,  800  Hz 

Point  C  (foot) 

35,  150,  300,  650  Hz 

Compact  actuator 

Point  A  (hip) 

1510,  1810  Hz 

Point  B  (knee) 

800  Hz 

Point  C  (foot) 

30,  150,250,  650,  1150,  1450  Hz 

changes  the  associated  tip  displacements  from  the  ideal  case. 
Similarly,  in-plane  motions  of  the  two  lateral  actuator  arrays 
(x,  y,  and  Oz  displacements)  are  very  similar  to  experimental 
results,  so  long  as  the  interconnect  stiffness  is  accounted  for  in 
the  design  as  fabricated.  With  the  existing  robot  leg  configu- 
ration, this  means  that  substantial  in-plane  rotations  can  only 
be  generated  by  one  joint  at  a  time,  although  both  joints  were 
shown  to  function.  The  total  in-plane  displacement  of  the  foot 
was  approximately  900  /im  for  a  4.5-mm-long  leg. 

D.  Frequency  Response 

Resonance  analysis  of  the  fabricated  m-DoF  microrobotic 
legs  indicates  that  such  a  robotic  appendage  is  capable  of 
moving  under  high  bandwidth.  Fig.  11  shows  the  frequency 
response  of  point  C  measured  with  respect  to  the  global  co- 
ordinate of  high-speed  camera  under  stereoscope  when  the 
vertical  actuator  array  is  activated  with  the  driving  sinusoidal 
input  voltage  where  the  offset  and  peak-to-peak  voltages  are 
chosen  to  3  V  and  2  Vpp,  respectively.  Q  factor  along  the 
vertical  motion  is  approximately  obtained  as  8.58  at  the  first 
resonant  frequency  of  34.7  Hz  in  a  ten-base  logarithm  scale. 
Table  VI  shows  resonant  frequencies  of  other  points  when 
vertical  actuator  and  compact  actuator  sets  are  independently 
activated.  When  the  vertical  actuator  is  activated,  a  280-ms 
settling  step  time  was  observed  at  the  foot. 

E.  Weight  and  Power  Considerations 

For  use  in  producing  terrestrial  microrobot  locomotion, 
weight  bearing  and  energy  use  of  a  given  robot  leg  design 
are  critical.  As  fabricated,  the  primary  source  of  out-of-plane 
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compliance  to  load  at  the  leg  tip  is  the  vertical  actuators. 
This  compliance  can  be  estimated  from  (3),  using  an  external 
moment  on  the  vertical  actuator  equal  to  the  weight  load  times 
the  leg  length.  Estimated  compliance  is  3080  rad/N.  For  the 
actuated  motion  of  the  foot  to  be  greater  than  any  bending 
of  the  vertical  actuators  during  load  bearing,  this  produces  an 
absolute  maximum  weight  capacity  of  2. 1  mg  per  foot,  which 
is  somewhat  further  reduced  by  out-of -plane  compliance  of  the 
lateral  actuator  arrays.  Fortunately,  each  actuator  array  has  a 
capacitance  of  only  1.2  nF,  such  that  power  consumption  of 
three  actuators  at  15  V  with  a  15 -Hz  step  frequency  is  only 
12  /iW.  This  would  correspond  to  an  approximate  speed  of 
27  mm/s  for  the  observed  0.9-mm  step  lengths  of  15  steps/s 
from  each  leg.  This  translates  to  an  ability  to  operate  with 
battery  power  densities  as  low  as  5.7  W/kg,  well  below  that 
of  current  battery  technologies.  In  practice,  the  ability  to  suc- 
cessfully operate  a  robot  leg  of  this  type  with  a  power  supply 
that  it  can  carry  is  more  dependent  on  the  efficient  use  of  energy 
in  any  control  and  energy  conversion  circuitry,  than  on  its  own 
fundamental  power  consumption  [15]. 

Still,  for  the  immediate  development  of  microrobotic  pro- 
totypes, the  current  work's  primary  benefits  are  the  increased 
reliability  of  thin-film  PZT  actuator  arrays  and  verification 
of  actuator  array  modeling  techniques.  This  process  has  pro- 
duced a  prototype  hexapod  using  the  leg  design  analyzed  that 
has  been  produced  with  five  out  of  six  legs  that  are  intact 
(one  leg  failing  due  to  fracture  of  a  flexure/tether  unit  during 
fabrication).  Moving  forward,  however,  more  feasible  robot 
prototypes  for  the  next  phase  of  further  robot  development 
will  utilize  millipede- style  layouts,  with  larger  numbers  of  legs, 
fewer  actuators  per  leg,  and  shorter  distances  to  leg  tips  to 
be  explored.  These  prototypes  thus  trade  reduced  mobility  for 
larger  weight-bearing  capacity  (up  to  approximately  200  mg 
for  an  entire  chassis)  to  allow  greater  flexibility  in  control  and 
power  systems.  Nonetheless,  the  ability  to  create  as  complex  leg 
geometries  as  described  here  while  meeting  fundamental  needs 
for  energy  use  versus  weight-bearing  capacity  and  the  ability 
to  predict  leg  behavior  given  fabrication  constraints  appear 
encouraging  for  a  long-term  goal  of  highly  mobile  bioinspired 
microscale  robots. 

V.  Conclusion 

Prototype  microrobotic  leg  joint  arrays  based  on  thin-film 
piezoelectric  actuation  are  described,  fabricated,  and  charac- 
terized in  this  paper.  These  leg  joints  take  advantage  of  high 
electric  fields  and  good  piezoelectric  coefficients  for  thin-film 
PZT  to  generate  large  forces  and/or  joint  angles  from  in-plane 
or  out-of -plane  motion.  They  are  coupled  with  a  robust  fab- 
rication process  that  allows  multiple  joints  to  be  incorporated 
into  full  high-mobility  legs.  Modeling  of  thin-film  PZT  lateral 
and  vertical  actuator  arrays  is  described,  noting  the  benefits  of 
having  both  free  PZT  unimorphs  and  thicker  silicon  structures 
in  a  combined  structure.  This  integration  is  done  with  com- 
plex high-aspect-ratio  silicon  structures  encapsulated  by  silicon 
dioxide  vertical  barrier  trenches.  Consistent  undercut  length  of 
thin-film  PZT  actuator  array  is  obtained  with  minimal  effect  on 
the  piezoelectric  performance  of  the  actuators. 


In  experimental  validation,  both  types  of  joint  are  shown 
to  function  in  the  integrated  structure,  with  angular  displace- 
ments varying  from  joint  to  joint.  The  in-plane  motion  gen- 
erated by  the  final  leg  structure  is  smaller  than  the  intended 
motion  generated  by  the  proposed  m-DoF  microrobotic  leg 
due  to  the  remaining  silicon  dioxide  vertical  barrier  trenches. 
Nonetheless,  performance  is  well  predicted  by  microrobotic 
joint  modeling  that  includes  the  following:  1)  compensation 
for  intrinsic  residual  stress  of  thin-film  stacks  that  changes  the 
initial  deflection  of  the  actuator  arrays;  2)  behavior  of  in-plane 
flexure  joint  array  structures  when  the  width  of  oxide  trench 
barrier  is  taken  into  consideration;  and  3)  design  of  an  elec- 
trical interconnect  structure  connected  with  the  outer  compact 
actuator  array  with  less  impact  on  the  final  device  motion. 
Vertical  actuation  performed  near-original  design  expectations 
(~40°)  with  only  the  neutral  position  changed  by  residual 
stress. 

The  completed  leg  structures  permit  faster  leg  movement 
with  more  degrees  of  freedom  than  previous  walking  micro- 
robot  actuators.  Weight  bearing  is  small  but  sufficient  for  the 
power  demands  of  the  piezoelectric  actuators.  Further  improve- 
ment to  the  current  design  and  fabrication  of  thin-film  PZT  de- 
vices will  support  continued  microrobotic  chassis  development. 
This  includes  the  parametric  design  of  the  stiffness  of  silicon 
flexure  and  tether  in  which  the  oxide  vertical  barrier  trenches 
are  taken  into  account,  and  the  run-to-run  stress  analysis  and 
process  control  of  metal  stack  layers  by  which  the  initial  out- 
of-plane  bending  of  the  appendage  is  minimized. 

Appendix 

Lateral  Actuator  Array  Spring  Definitions 

From  (4),  the  rotational  spring  stiffness  values  of  the  flexure, 
tether,  and  actuator  art  kf^o,  kt^e,  and  ks,ct,9,  respectively 

J  (^^)flex  .... 

^flex 

kt,e  =  r   ^1^) 

Kct,o=Kct{F^ct)Ll^t  (13) 

where  /cact  is  the  axial  spring  stiffness  of  the  actuator  dependent 
on  the  actuation  force,  (^/)fiex  and  {EA)t  are  the  composite 
flexural  rigidity  values  of  the  silicon  and  silicon  dioxide  flexure 
and  tether  along  transverse  and  axial  directions,  respectively, 
Lact  is  the  distance  between  the  tether  and  the  flexure  joint, 
and  Lfiex  and  Lt  are  the  lengths  of  the  flexure  joint  and  tether 
as  shown  in  Fig.  1  and  Table  I.  Because  the  actuator  itself 
and  the  silicon  tether  experience  the  axial  force  generated  by 
piezoelectric  film,  the  maximum  rotary  angle  is  represented  in 
terms  of  the  combined  spring  stiffness  of  the  joint  structure  in 
(11)-(13)  as  shown  in  Fig.  3. 

Lateral  Array  Modifications  for  Interconnect  Stiffness 

When  combining  lateral  actuator  models,  such  as  that  defined 
by  (4)  and  (11)-(13),  basic  equation  (4)  can  be  applied  to  the 


56 


outer  in-plane  actuator  array  (joint  7),  but  in  the  m-DoF  device 
fabricated,  the  motion  of  the  inner  lateral  actuation  array  (joint 
^)  is  limited  by  the  need  to  deliver  an  electrical  signal  to  joint 
7.  This  signal  is  transferred  via  an  elastic  interconnect  created 
from  the  PZT-metal  stack.  As  the  inner  compact  array  and 
the  interconnect  are  mechanically  connected  in  parallel,  the 
following  relation  is  used  to  predict  in-plane  motion  of  the 
resulting  combination,  based  on  the  assumption  that  only  small 
angular  displacements  contribute  nonnegligible  stiffness  to  the 
elastic  interconnect: 


(14) 


where  (EI)[c,z  and  Lie  are  the  composite  in-plane  rigidity  and 
effective  length  of  the  material  stacked  PZT  thin-film  intercon- 
nect and  M[c,z  is  the  moment  applied  at  the  interconnect  by 
the  inner  compact  actuators.  The  displacement  of  the  tip  of  the 
elastic  interconnect  from  the  center  of  in-plane  compact  array 
6c  is  estimated  from  the  length  of  the  PZT  interconnect  and  the 
sum  of  rotational  displacements  of  each  link  T^{Oz^(p))-.  Since 
the  inner  compact  array  and  the  PZT  thin  film  are  connected 
in  parallel,  the  radius  of  rotation  exerted  by  inner  compact 
array  is  close  to  5c  for  the  small  displacement  exerted  by  the 
parallel  structure  of  the  interconnect  and  inner  compact  array. 
The  moment  is  equivalent  to  the  force,  which  is  the  ratio  of 
Mic^z  to  6c  from  the  center  of  rotation,  applied  at  the  tip  of 
eighth  link  of  inner  compact  array.  This  force  applied  at  eighth 
Hnk  is  then  projected  onto  each  flexure  by  /f^  at  ith  link 
in  addition  to  the  forcing  component  by  individual  actuators  to 
form  new  forcing  terms  within  the  array 


cos  (6>^,(^))g 
0 


sm  (^e^^,(/3)j 
cos  (6>z,(/3)) 

0 


(15) 


where  g  is  the  sum  of  rotational  displacements  from  ith 

link  to  eighth  link.  Then,  the  forcing  term  f^^^  in  (5)  consists  of 

[^act,  0,  ^act]^  and  /f^,  and  the  nonlinear  system  of  equations 
from  (5),  (14),  and  (15)  is  solved  numerically  for  {Oz^^p)).  [9]. 


From  (10),  the  stiffness  matrix  Kic,eq  for  PZT  interconnect 
is  modeled  as 


"^ic.eq^ic 


0 
0 
0 
0 

F- 

ic,: 
Mi,. 


0 
0 

-Ll{2{EI),c,y)-' 
0 


0 

0 
0 

ic{EI) 


1 


(16) 


where  Fic  is  the  forcing  term  applied  at  the  tip  of  the  intercon- 
nect with  the  axial  force  F^c^x^  out-of-plane  moment  M^c^y,  and 
rotational  moment  Mic,^,  respectively.  Then,  the  relation  of  the 
forcing  term  Fic  to  the  lateral  actuation  force  Fact  within  joint 
P  is  obtained 


^act^^c  ^  cos  ( 


^act^dist^c^^c^COS  (  {^z,{/3))i 
\i=l 


N  .  Fact 

(17) 


A  lumped  compliance  matrix  for  the  joint  /3  is  derived 
for  the  case  when  only  very  small  displacements  of  point  B 
are  observed  and  joint  displacement  is  assumed  to  be  exerted 
equally  by  each  actuator.  This  model,  combining  the  motions 
at  the  eight  individual  actuators  into  an  approximate  lumped 
model,  becomes  (18),  shown  at  the  bottom  of  the  page. 
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Abstract — ^As  the  capability  and  complexity  of  robotic  plat- 
forms continue  to  evolve  from  the  macro  to  the  micron  scale, 
the  challenge  of  achieving  autonomy  requires  the  development  of 
robust,  lightweight  architectures.  These  architectures  must  pro- 
vide a  platform  upon  which  actuators,  control,  sensing,  power, 
and  communication  modules  are  integrated  for  optimal  perfor- 
mance. In  this  paper,  the  first  autonomous  jumping  microrobotic 
platform  is  demonstrated  using  a  hybrid  integration  approach  to 
assemble  on-board  control,  sensing,  power,  and  actuation  directly 
onto  a  polymer  chassis.  For  the  purposes  of  this  paper,  jumping 
is  defined  as  brief  parabolic  motion  achieved  via  an  actuation 
pulse  at  takeoff.  In  this  paper,  the  actuation  pulse  comes  from 
the  rapid  release  of  chemical  energy  to  create  propulsion.  The 
actuation  pulse  lasts  several  microseconds  and  is  achieved  using  a 
novel  high-force/low-power  thrust  actuator,  nanoporous  energetic 
silicon,  resulting  in  250  /la  J  of  kinetic  energy  delivered  to  the  robot 
and  a  vertical  height  of  approximately  8  cm.  [2011-0030] 

Index  Terms — Autonomy,  microrobot,  porous  silicon. 

1.  Introduction 

SMALL  autonomous  robotic  platforms  hold  the  potential  to 
help  target  the  efforts  of  first  responders,  provide  stealthy 
surveillance,  or  add  mobility  to  sensor  networks.  However, 
existing  microrobot  platforms  have  limited  autonomy  [1]-[11]. 
Autonomous  microrobot  platforms  require  essential  compo- 
nents including  sensors,  actuators,  electronic  circuits,  and  a 
power  supply.  These  components  must  further  be  integrated 
to  provide  certain  essential  functions  including  conversion  of 
sensor  data  to  actionable  information,  intelligence  to  make 
decisions  based  on  the  information,  and  mobility  to  take  action 
on  the  decisions.  Integration  of  sensing,  control,  power,  and 
actuation  on  a  single  chassis  increases  the  utility  of  robots, 
allowing  them  to  be  placed  in  environments  where  they  can 
sense,  think,  and  act  with  limited  or  no  human  intervention 
[l]-[5].  These  enabling  features  of  autonomy  lead  to  robots  that 
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will  eventually  traverse  environments  unreachable  by  humans, 
while  providing  critical  data,  and  the  ability  to  adapt  to  ever 
changing  surroundings. 

As  robot  length  scales  decrease  below  the  centimeter  scale,  it 
becomes  an  increasing  challenge  to  integrate  systems  to  deliver 
complex  functionality  on  a  single  platform.  Larger  robots  like 
RoACH  [6]  or  Alice  [7]  are  assembled  using  commercial  off- 
the-shelf  (COTS)  components  including  motors,  sensors,  and 
electronics  with  little  difficulty.  Application-specific  integrated 
circuits  (ASICs)  provide  a  unique  approach  to  developing  con- 
trol architectures  for  millimeter  scale  platforms  by  reducing  the 
chip  area  [5].  This  paper  will  take  a  new  hybrid  integration 
approach  that  includes  both  custom  fabrication  (similar  to  the 
bottom-up  design  of  ASICs)  and  COTS  components  (similar  to 
simple  assembly  available  to  larger  robots). 

Recent  work  in  microrobotics  has  made  progress  toward 
achieving  autonomous  functionality,  which  the  authors  define 
as  the  integration  of  sensing,  actuation,  power,  and  control  on  a 
single  chassis.  These  characteristics  are  available  to  the  robot 
independent  of  the  operating  environment.  A  number  of  the 
published  microrobots  can  be  characterized  as  "actuator  only" 
platforms  because  they  rely  on  a  controlled  operating  environ- 
ment to  function,  cannot  make  decisions  based  on  the  environ- 
ment, and  cannot  carry  their  power  supply.  The  microrobot  in 
[8]  maneuvers  using  an  integrated  scratch  drive  actuator  and  a 
cantilever  steering  arm,  allowing  it  to  achieve  speeds  greater 
than  200  /im/sec.  The  operation  of  the  microrobot  depends  on 
an  underlying  electrical  grid,  which  provides  power  and  control. 
The  "walking  chip"  in  [9],  designed  to  carry  a  maximum 
external  load  of  2.5  g  on  its  back,  relies  on  external  power  in 
the  form  of  tethered  wires  that  measure  up  to  10  cm  in  length 
and  lacks  sensing  and  on-board  control.  The  bio-inspired  fly  in 
[10]  demonstrates  insect-like  wing  trajectories  with  integrated 
actuation  and  mechanics,  but  does  not  yet  possess  integrated 
on-board  sensing,  power,  or  control. 

The  complexity  of  the  design  and  integration  increases  as 
more  functionality  is  incorporated  directly  onto  the  chassis. 
The  microrobot  in  [11]  appears  to  come  close  to  our  definition 
of  autonomy  using  solar  cells  for  power,  electrostatic  motors 
for  actuation,  and  a  CMOS  finite  state  machine  for  control. 
These  components  are  fabricated  individually  and  integrated 
using  a  hybrid  approach  involving  wirebonding  to  electrically 
connect  each  component.  However,  the  robot  does  not  have 
integrated  sensors  to  allow  it  to  observe  and  react  to  changes 
in  the  environment.  The  only  robots  that  appear  to  include 
all  components  necessary  for  individual  autonomy  are  several 
centimeters  in  size  or  larger  [6],  [12]. 
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Fig.  1 .  Computer  model  of  jumping  microrobot  showing  the  polymer  chassis 
and  control  circuit.  The  energetic  thrust  actuator  is  underneath,  on  the  "belly" 
of  the  robot. 

While  these  microrobots  have  demonstrated  ingenious  func- 
tionahty,  the  goal  remains  to  develop  a  platform  that  is  truly 
autonomous,  where  all  required  functionality  is  provided  on- 
board the  microrobotic  platform.  To  overcome  the  challenges 
associated  with  implementation  and  integration  on  a  sub- 
centimeter  scale,  the  authors  have  developed  a  novel  hybrid 
integration  approach  to  fabricate  a  microrobotic  platform, 
conceptually  shown  in  Fig.  1.  The  microrobotic  platform  is 
designed  with  a  polymer  chassis,  upon  which  micron-scale 
electrical  traces  are  lithographically  defined  and  patterned  to  in- 
tegrate power  (capacitors),  sensing  (a  phototransistor),  and  the 
most  rudimentary  form  of  decision  making  (a  single  transistor). 
Nanoporous  energetic  silicon,  a  solid  energetic  formulation  that 
produces  gas  when  ignited  [13],  [14],  functions  as  a  thruster  to 
provide  mobility.  The  nanoporous  energetic  silicon  is  mounted 
underneath  the  polymer  chassis.  The  microrobotic  platform  can 
detect  a  change  in  the  ambient  light  and  respond  to  the  change 
by  triggering  ignition  of  the  energetic  material.  This  produces 
an  upward  thrust. 

This  work  offers  two  key  contributions  toward  greater  au- 
tonomy in  microrobotics  along  with  a  demonstration  of  this 
autonomy.  First,  a  novel  energetic  silicon-based  actuator  is 
proposed  to  enhance  mobility  (Section  II).  Mobility  is  demon- 
strated in  the  form  of  vertical  thrust  as  stored  chemical  energy 
is  rapidly  converted  to  mechanical  energy.  While  only  a  single 
propulsion  event  is  demonstrated,  the  nanoporous  energetic 
silicon  can  be  arrayed  on-chip  to  provide  multiple  propulsion 
events  in  different  directions.  The  second  contribution  develops 
a  hybrid  integration  approach  used  to  mechanically  and  electri- 
cally integrate  this  actuator  with  sensing  and  control  elements 
(Section  III).  The  integration  of  on-chip,  chemical-based  actu- 
ation with  electronic  controls  results  in  sufficient  autonomy  for 
applications  ranging  from  mobile  sensor  networks  to  dynam- 


TABLE  I 

Physical  Dimensions  of  System  Components 
Used  to  Design  the  Hexapod 


Component 

Mass  (mg) 

Footprint 
Dimensions 

1  OuF  cap 

9 

0.8  X  1.6  mm^ 

lOOuF  cap 

133 

2.6x3.4  mm^ 

FET 

8 

2.9  X  1.3  mm^ 

Phototransistor 

3 

1 .2  X  2.0  mm^ 

1  Ok  resistor 

5 

l.SxO.Smm^ 

Chassis  w/ alloy 

62-69 

4.0  X  7.0  mm^ 

Silicon  chip 

22-28 

4.0  X  4.0  mm^ 

Metal  pads 

14-18 

n/a 

Wires 

11-15 

n/a 

Extra  polymer 

13-36 

n/a 

ically  controlled  microthrusters.  These  two  contributions  are 
combined  to  demonstrate  the  first  robot  at  this  size  scale  with 
the  ability  to  sense,  think,  and  act  (Section  IV). 

II.  System  Architecture 

The  goal  for  the  microrobot  system  architecture  was  to 
develop  a  robotic  platform  that  could  sense  a  change  in  light 
intensity  and  respond  by  jumping.  To  simplify  matters,  the 
microrobot  would  jump  vertically  with  a  takeoff  angle  of  90° 
rather  than  follow  a  particular  direction.  To  accomplish  this 
goal,  the  system  architecture  required  a  sensor,  control  circuit, 
actuator,  and  power,  each  of  which  needed  to  be  integrated  on  a 
polymer  chassis.  The  electronics  were  designed  and  fabricated 
with  COTS  parts  to  provide  both  on-board  power  and  logic  to 
sense  a  change  in  light  intensity  and  to  provide  an  electrical 
stimulus  to  trigger  the  energetic  material.  Table  I  summarizes 
the  size  and  weight  of  each  of  the  components  of  the  system, 
and  the  following  subsections  describe  in  detail  the  function 
of  each  part.  The  total  mass  of  the  assembled  system  was 
approximately  300  mg.  In  our  experiments,  three  hexapods 
were  assembled  and  tested.  The  overall  mass  of  the  hexapod 
varied  from  280  mg  to  3 1 8  mg  across  the  platforms  that  were 
assembled.  The  largest  contributor  to  this  variation  in  mass  was 
an  additional  layer  of  polymer  that  was  applied  to  the  hexapod 
once  it  was  assembled  to  encapsulate  the  IC  components  and 
increase  the  robustness  of  the  robot. 

A.  Actuation 

Instead  of  using  a  mechanical  spring  to  store  energy  as  seen 
in  [15],  nanoporous  energetic  silicon  was  chosen  to  store  and 
release  energy  chemically.  The  primary  benefit  to  this  approach 
is  reduced  complexity  and  lower  part  count  due  to  eliminat- 
ing the  need  for  a  separate  electrically  controlled  actuator  to 
compress  a  spring.  Nanoporous  energetic  silicon  is  formed 
in  an  electrochemical  etch  process  as  outlined  in  [13].  It  is 
initially  inert,  and  only  becomes  energetic  when  infused  with  an 
oxidizer  such  as  sodium  perchlorate.  The  exothermic  reaction 
can  be  triggered  with  heat,  friction,  or  focused  light.  Multiple 
actuators  can  be  integrated  onto  the  chassis  of  a  microrobot, 
allowing  it  to  achieve  multiple  jumps  in  different  directions. 
For  this  work,  actuation  was  demonstrated  using  a  single,  2  mm 
diameter  region  of  nanoporous  energetic  silicon. 
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Fig.  2.  Tethered  jump  of  100-mg  hexapod  platform,  tethered  by  12-mil  wire. 
The  images  are  spaced  10  frames  (10  ms)  apart.  In  frames  1-3,  a  portion  of  the 
silicon  chip  detached  from  the  hexapod. 

Propulsion  of  a  hexapod  platform  was  demonstrated  using 
the  nanoporous  energetic  silicon.  Fig.  2  shows  an  initial  (teth- 
ered) experiment  with  a  2  mm  diameter,  40  /xm  thick  volume 
of  nanoporous  energetic  silicon  incorporated  on  the  polymer 
hexapod  chassis  [16].  The  nanoporous  silicon  was  attached 
to  the  underside  of  the  hexapod,  and  connected  via  wires  to 
an  external  DC  power  supply.  The  components  of  the  control 
circuit  were  not  attached  to  the  hexapod  in  this  experiment, 
resulting  in  a  significantly  lower  system  mass.  Including  the 
wire  leads  and  the  energetic  silicon  chip,  the  hexapod  weighed 
100  mg  in  this  configuration.  The  hexapod  travelled  up  out  of 
the  ~11  cm  field  of  view.  The  takeoff  velocity  was  measured 
at  2.8  m/s  from  high-speed  video  frames  captured  at  1000 
frames/s.  Based  on  this,  the  projected  jump  height  was  40  cm. 
This  experiment  showed  that  nanoporous  energetic  silicon  is  a 
viable  actuator  for  a  jumping  robot,  so  the  authors  proceeded  to 
integrate  sensing,  control,  and  power  functions  onto  the  chassis. 

B.  Sensing  and  Control 

The  microrobotic  platform  is  designed  to  jump  in  response 
to  "seeing"  a  change  in  light  level,  as  an  insect  which  may 
jump  in  response  to  sensing  the  shadow  of  a  predator.  Sensing 
is  achieved  using  a  light-sensitive  phototransistor  (SFH3710) 
shown  in  Fig.  3.  The  space  available  to  integrate  electronic 
control,  sensing,  and  actuation  is  limited  to  the  top  and  bottom 
of  the  chassis,  approximately  56  mm^  total  area.  The  top  of 
the  chassis  is  used  to  integrate  a  circuit  composed  of  five 
discrete  surface-mount  components  (shown  in  Fig.  3),  while  the 
underside  of  the  chassis  houses  the  energetic  material  used  for 
actuation.  The  control  circuit  itself  consists  of  a  2N7002,  single 
n-channel  enhancement  MOSFET,  and  a  10  kO  resistor  used  to 
bias  the  gate  of  the  MOSFET. 

The  spectral  sensitivity  of  the  NPN  phototransistor  chosen 
is  350  nm  to  950  nm  with  a  maximum  sensitivity  at  570  nm. 
When  the  lighting  condition  is  varied  from  an  illuminance 
of  ~0.3  lux,  which  is  comparable  to  a  full  moon  on  a  clear 
night  [17],  to  approximately  26  lux,  somewhat  less  than  the 
illuminance  of  a  family  room  [18],  the  voltage  at  the  gate  of 
the  MOSFET  rises  to  5  V.  This  closes  the  MOSFET  transistor. 


Fig.  3.  Circuit  designed  to  provide  sensing  and  actuation.  The  3  VL  resistor 
represents  the  hotwire  used  to  ignite  the  energetic  material. 

A  series  connection  of  a  10  /iF  capacitor  and  3  O  resistor  with 
the  MOSFET  is  shown  in  Fig.  3.  The  3  O  resistor  is  used  to 
model  the  electrical  characteristics  of  an  electrothermal  initiator 
used  to  ignite  the  energetic  material.  Details  about  the  electrical 
initiator  design  and  performance  are  discussed  in  [13].  The 
10  /iF  capacitor  is  precharged  to  6  V  for  the  experiments 
discussed  here.  As  the  MOSFET  closes,  it  provides  a  path  for 
current  to  flow  through  the  electrical  initiator,  producing  heat 
to  ignite  the  energetic  material.  The  electrothermal  initiator  re- 
quires at  least  150  mA  to  ignite  the  energetic  material.  However, 
it  should  be  noted  that  this  current  is  only  required  for  at  most 
100  /iS.  In  this  system,  the  capacitor  discharges  directly  through 
the  electrothermal  initiator  wire,  without  voltage  or  current 
regulation.  This  arrangement  simplifies  the  circuit,  reduces  the 
part  count,  and  is  sufficient  to  ignite  the  energetic  silicon. 

C.  Power 

The  two  capacitors  shown  in  Fig.  3  provide  power  to  the 
phototransistor  and  the  initiator  on  the  nanoporous  energetic 
silicon.  The  100  ii¥  capacitor  provides  power  to  bias  the 
phototransistor.  The  10  /iF  capacitor  is  used  to  provide  power  to 
ignite  the  nanoporous  energetic  silicon  once  the  phototransistor 
detects  the  increase  in  light  intensity.  Both  the  100  /iF  and 
10  /iF  capacitors  in  Fig.  3  are  precharged  to  6  V.  Both  power 
sources  are  isolated  until  an  event  is  detected,  triggering  the 
actuation  response.  A  larger  capacitor  is  chosen  to  power  the 
phototransistor  to  increase  the  time  constant  and  supply  a  larger 
amount  of  energy  to  accommodate  leakage.  While  the  longevity 
of  the  capacitors  as  power  sources  is  not  investigated  in  detail, 
the  100  /iF  capacitor  is  able  to  retain  sufficient  charge  to  power 
the  phototransistor  for  up  to  8  min  in  a  dimly  lit  room  (^0.3 
lux).  While  the  expected  discharge  time,  based  on  the  off-state 
resistance,  for  the  phototransistor  and  the  FET  is  approximately 
40  min,  a  significant  amount  of  leakage  current  across  the 
phototransistor  results  from  exposure  to  the  low-level  ambient 
light.  It  should  be  noted  that  the  100  /iF  capacitor  used  to 
bias  the  phototransistor  is  the  heaviest  single  part  of  the  robot 
(Table  I),  with  the  much  smaller  capacitor  only  acting  to  ini- 
tiate the  energetic  material.  In  the  future,  the  requirement  for 
this  larger  capacitor  may  be  removed  through  better  sensors 
or  through  the  use  of  a  low-volume,  high-capacity  thin  film 
battery.  The  energy  associated  with  the  capacitors  and  the 
nanoporous  energetic  silicon  is  summarized  in  Table  II. 


TABLE  II 

Summary  of  Energy  Supplied  by  the  Capacitors  and 
Energy  Supplied  to  and  Generated  by  the 
Nanoporous  Energetic  Silicon 


Component 

Type  of  Energy 

Energy 

10  }iF  Capacitor 

Electrical 

0.18  mJ 

100  |iF  Capacitor 

Electrical 

1.8  mJ 

Energetic  Porous  Si 

Electrical  (ignition) 

0.18  mJ 

Energetic  Porous  Si 

Chemical 

1.9  J 

III.  Hybrid  Integration 

To  electrically  and  mechanically  integrate  the  sensing, 
power,  and  control  components  with  the  actuator  and  robot 
chassis  with  minimal  added  mass,  a  new  hybrid  integration 
process  was  designed.  This  approach  used  a  polymer  chassis 
for  structural  support  to  carry  a  payload  as  well  as  a  substrate 
for  electrical  traces  for  the  integration  of  the  control  circuit. 

A.  Polymer  Chassis 

The  hexapod  style  chassis  was  chosen  to  enable  walking  in 
future  iterations  of  the  platform,  and  the  polymer  was  chosen 
for  robustness,  durability,  light  weight,  and  ease  of  fabrication. 
The  chassis  provided  the  robot's  functional  skeleton  as  well  as 
a  point  of  assembly  on  which  to  integrate  all  other  components 
in  this  hybrid  integration  approach  (Fig.  1).  It  is  4  mm  by  7  nrni 
by  0.5  mm  in  dimensions  and  was  fabricated  using  a  rapid  pro- 
totyping process  [19].  A  photodefinable  polymer  (Loctite3525 
modified  acrylic)  was  first  spread  evenly  on  a  transparent  plastic 
sheet,  and  another  transparent  sheet  was  placed  on  top  of  the 
polymer  film.  Glass  slides  were  placed  between  the  two  sheets 
to  define  the  thickness.  A  photolithographic  mask,  with  the 
hexapod  features  defined  as  clear  windows  in  an  opaque  field, 
was  then  placed  on  top,  and  UV  light  was  used  to  expose  the 
hexapod  features  through  the  mask.  The  exposed  polymer  was 
hardened,  and  the  unexposed  polymer  could  then  be  removed 
with  a  methanol  rinse.  The  metal  traces  for  assembly  of  the 
control  circuit  were  deposited  and  patterned  (described  in  the 
following  section),  then  the  legs  were  bent  out  of  plane,  and  an 
additional  layer  of  polymer,  [shown  in  Fig.  4(c)],  was  applied  to 
the  inside  corner  between  the  legs  and  the  body  and  cured.  This 
resulted  in  the  3-D  hexapod  structure  similar  to  that  in  Fig.  4(d), 
which  allows  for  future  integration  of  leg  actuators  for  bimodal 
(walking/jumping)  mobility  or  to  orient  the  robot  for  a  jump  in 
a  particular  direction. 

B.  Metallization 

A  copper  film  was  deposited  on  top  of  the  polymer  chassis 
using  metal  evaporation  at  room  temperature.  Once  deposited, 
the  electrical  traces  were  lithographically  patterned  on  the 
copper  film.  With  photoresist  protecting  the  underlying  copper 
metal  traces,  the  exposed  copper  was  etched  using  a  2 : 1 
mixture  of  water  to  nitric  acid.  To  facilitate  attaching  the  circuit 
elements  to  a  flexible  substrate,  the  copper  traces  were  then 
coated  with  a  low  melting  temperature  solder  (Indalloy  117 
from  Indium  Corporation)  using  a  dipping  process  discussed 


Fig.  4.  Process  flow  to  fabricate  3-D  hexapod  structure.  This  process  flow 
shows  metal  patterned  on  the  legs.  For  the  purpose  of  our  experiment,  metal 
was  only  patterned  on  the  body. 


Fig.  5.  Metalized  layer  begins  to  crack  when  submerged  in  low-temperature 
solder.  Copper  traces  were  subsequently  widened  to  472  fim  to  allow  enough 
solder  to  heal  the  cracks. 


in  [20].  Indalloy  117  melts  at  47  °C,  and  was  chosen  because 
the  Loctite  polymer  softens  above  100  °C  and  using  a  solder 
iron  to  heat  the  solder  would  result  in  damage  to  the  polymer 
substrate.  The  circuit  components  can  be  attached  by  reflowing 
the  low-temperature  solder  without  damaging  the  substrate. 
For  this  process,  the  solder  was  kept  at  65  °C  to  ensure  the 
robustness  of  the  polymer  chassis.  A  mixture  of  ethylene  glycol 
and  hydrochloric  acid  was  added  to  the  solder  dipping  crucible 
to  remove  any  surface  oxides  on  the  copper  traces  and  on  the 
solder.  The  polymer  chassis  was  slowly  submerged  into  the 
molten  alloy,  and  retracted  after  10  s.  About  50%  of  the  time,  a 
visual  inspection  revealed  incomplete  coating,  and  the  chassis 
was  submerged  again  to  coat  all  copper  traces.  Initial  results 
indicated  a  tendency  for  the  copper  layer  to  crack  when  cooling 
after  removal  from  the  solution,  as  shown  in  Fig.  5.  Therefore, 
the  minimum  copper  feature  size  was  increased  from  133  /im 
to  472  /im,  providing  greater  surface  area  to  capture  enough 
solder  to  "heal"  any  cracks  that  form. 
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Fig.  6.  Metal  bond  pads  fixed  to  Si  allow  the  energetic  to  be  connected  to  the 
control  circuit. 

C.  Assembly 

A  serial  process  was  used  to  assemble  each  of  the  compo- 
nents. The  hexapod  and  circuit  components  were  separately 
coated  with  the  low  melting  temperature  alloy.  A  solder  iron 
was  used  to  manually  solder  external  wires  to  the  capacitors 
and  the  MOSFET  before  the  components  were  assembled  onto 
the  hexapod  chassis.  This  allowed  electrical  leads  to  be  easily 
attached  to  charge  the  capacitor  power  sources  after  assem- 
bly. The  polymer  hexapod  was  then  placed  on  a  hotplate  at 
65  °C  to  melt  the  Indalloy,  and  the  circuit  components  were 
positioned  using  tweezers.  Moderate  pressure  was  applied  man- 
ually between  the  component  and  the  binding  site  to  ensure 
a  robust  connection.  The  hexapod  was  then  removed  from 
the  hotplate  and  allowed  to  cool.  As  the  polymer  substrate 
cooled,  the  circuit  components  were  fused  to  the  binding 
sites. 

Attempts  to  solder  directly  to  the  energetic  porous  silicon 
chip  were  unsuccessful  because  the  thin-film  initiator  was  not 
robust  enough  to  withstand  the  heat  of  a  soldering  iron.  The  low 
melting  temperature  alloy  was  not  attempted  because  of  con- 
cern that  the  dipping  process  might  contaminate  the  nanoporous 
silicon.  Although  it  was  possible  to  wirebond  to  the  silicon  chip, 
wirebonding  to  the  polymer  chassis  was  not  successful  because 
the  polymer  was  too  soft.  Therefore,  gold-plated  metal  pads 
shown  in  Fig.  6  were  glued  to  the  silicon  chip,  and  wirebonds 
were  made  between  the  integrated  initiator  and  the  metal  pads. 
Wires  were  then  soldered  between  the  control  circuit  and  these 
metal  pads,  and  the  nanoporous  energetic  silicon  was  attached 
to  the  underside  of  the  hexapod  using  a  double-adhesive  tape. 
In  future  iterations,  the  pads  could  be  eliminated  by  creating 
vias  in  the  polymer  chassis  and  the  nanoporous  silicon  substrate 
and  filling  them  with  metal  for  direct  connection  using  the  low- 
temperature  solder. 

Once  the  hexapod  was  assembled,  the  circuit  and  underlying 
electrical  traces  were  coated  with  an  additional  layer  of  Loctite 
to  prevent  delamination  of  the  rigid  circuit  elements  from  the 
flexible  polymer  skeleton.  This  robust  integration  allowed  the 
hexapod  legs  to  be  flexed  and  potentially  actuated  out  of  plane, 
without  disturbing  the  functionality  of  the  circuit.  This  unique 
process  therefore  uses  one  material  as  the  mechanical  structure, 
electrical  substrate,  and  packaging  material  for  the  microrobotic 


Fig.  7.  Metal  bond  pads  soldered  to  wires  coming  off  of  the  capacitor 
and  MOSFET. 

platform.  Fig.  7  shows  a  circuit  that  has  been  fully  assembled 
onto  the  polymer  chassis. 

D.  Packaging 

The  sodium  perchlorate  oxidizer  used  to  make  the 
nanoporous  silicon  into  an  energetic  material  is  strongly  hygro- 
scopic, and  the  absorption  of  water  by  the  oxidizer  degrades 
the  efficiency  of  the  energetic  reaction  and  produces  stress 
within  the  porous  layer.  The  induced  stress  can  also  cause  the 
porous  material  to  crack  and  the  hotwire  initiator  to  subse- 
quently break.  For  this  reason,  the  jumping  experiments  were 
all  conducted  in  a  dry  box  plumbed  with  nitrogen.  Even  so, 
the  process  of  opening  the  door  of  the  enclosure  to  charge  the 
capacitors  prior  to  ignition  allowed  enough  time  for  the  oxi- 
dized sample  to  absorb  water  and  degrade  the  efficiency  of  the 
reaction. 

Three  techniques  were  employed  to  limit  the  rate  at  which 
moisture  was  absorbed  by  the  oxidizer.  The  first  technique 
involved  applying  one  drop  of  oxidizer  solution  (3.2  M  NaC104 
in  methanol)  using  a  syringe  in  a  dry  room  controlled  to  <  1  % 
relative  humidity  and  allowing  the  sample  to  dry  for  20  min. 
After  the  sample  was  dried,  a  thin  layer  of  Loctite  (~500  /im 
thick)  was  carefully  applied  on  top  of  the  porous  silicon  and  UV 
cured  for  5  min.  The  second  technique  was  to  apply  the  oxidizer 
and  dry  in  a  nitrogen  dry  box,  then  seal  the  porous  silicon  with 
an  evaporated  parylene  layer  (^12  //m  thick).  While  loading 
the  samples  into  the  parylene  system,  there  was  some  exposure 
to  environmental  moisture.  An  alternative  approach  which  has 
not  been  attempted  yet  would  be  to  dry  the  sample  under 
the  dry  nitrogen  inside  the  parylene  coating  chamber  to  avoid 
transport. 

The  final  (and  simplest)  humidity  exposure  minimization  ap- 
proach was  to  connect  leads  to  the  hexapod  prior  to  application 
of  the  oxidizer  and  run  these  leads  outside  of  the  controlled 
environment  chamber.  The  oxidizer  was  allowed  to  dry  for 
15  min.  The  door  to  the  chamber  was  then  opened  very  briefly 
(less  than  30  s)  to  disconnect  the  leads  and  then  closed  for  the 
experiment. 

IV.  Results 

Experiments  were  performed  in  a  laboratory,  and  a  mod- 
erately dark  environment  was  simulated  by  turning  off  all 
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overhead  fluorescent  lights,  and  turning  on  a  small  12-W 
lamp  in  the  corner  of  the  room  to  allow  some  visibility 
for  positioning  the  device  and  connecting  and  disconnect- 
ing electrical  leads.  The  resulting  ambient  light  level  at  the 
photodetector  was  ^0.3  lux,  as  mentioned  above.  Oxidizer 
was  applied  to  the  nanoporous  energetic  silicon  using  one  of 
the  three  different  techniques  discussed  above  to  minimize 
moisture  exposure.  The  100  /jF  and  10  /iF  capacitors  were 
charged  to  6  V,  and  the  hexapod  was  positioned  approximately 
12  inches  under  a  high  intensity  3  LED  array  light  source.  The 
light  source  was  turned  on,  triggering  the  phototransistor,  which 
closed  the  MOSFET  switch,  which  in  turn  triggered  ignition  of 
the  energetic  nanoporous  silicon.  Each  jump  was  captured  by  a 
high-speed  camera  recording  at  1000  frames  per  second. 

The  first  untethered  experiment  was  performed  with  the 
Loctite  moisture  seal  encapsulating  the  porous  silicon.  The 
experiment  was  done  in  ambient  air;  therefore  an  encapsulation 
layer  was  needed  to  prevent  moisture  uptake  and  subsequent 
degradation  of  the  porous  silicon  and  oxidizer  mixture.  This 
hexapod  had  a  total  weight  of  280  mg,  and  achieved  a  vertical 
height  of  1  cm  when  the  light  was  turned  on.  The  limited 
height  may  have  been  due  to  packaging  the  sample  with  Loctite. 
Deposition  of  the  Loctite  directly  on  the  porous  silicon  resulted 
in  a  thick  layer  (^500  /im)  of  the  Loctite  remaining  on  the 
surface.  A  portion  of  the  output  energy  being  converted  into 
thrust  for  movement  would  therefore  be  lost  in  breaking  the 
seal  created  by  the  Loctite.  It  was  also  possible  that  the  Loctite 
coating  may  have  affected  the  oxidized  porous  silicon  surface, 
causing  it  to  be  less  reactive.  Similar  results  were  achieved  with 
the  parylene-coated  samples.  Further  work  must  be  done  to 
explore  other  alternatives  to  encapsulate  the  energetic  without 
degrading  the  performance. 

The  best  jump  heights  were  achieved  with  the  simplest 
humidity  exposure  minimization  approach,  which  involved  pre- 
connecting  the  electrical  leads  to  charge  the  capacitors  before 
applying  and  drying  the  oxidizer,  and  running  the  experiment 
in  a  nitrogen-filled  box.  The  box  was  only  very  briefly  opened 
(less  than  30  s)  to  disconnect  the  leads.  In  this  case,  a  314-mg 
hexapod  jumped  approximately  8  cm  vertically  (Fig.  8). 

The  authors  analyzed  approximately  200  frames  of  the  jump 
trajectory  to  determine  if  any  significant  acceleration  sources 
other  than  gravity  affected  the  microrobotic  platform  during 
flight.  Several  frames  at  the  end  of  the  trajectory  were  not 
captured  in  the  high-speed  video,  so  the  landing  portion  of  the 
trajectory  was  not  shown.  The  center  of  the  hexapod  chassis  was 
chosen  as  the  center  of  mass,  and  the  height  was  measured  as 
the  center  of  mass  crossed  each  half  centimeter  marker  on  the 
height  scale.  The  resulting  position  versus  time  data  is  shown 
in  Fig.  9.  Given  the  resolution  of  the  camera,  the  accuracy 
with  which  we  were  able  to  pinpoint  the  robot's  center  of  mass 
in  each  frame,  and  parallax  errors  involved  observation  from 
a  fixed  viewpoint,  we  estimated  the  uncertainly  for  each  data 
point  as  approximately  0.25  cm.  Using  this  uncertainty,  the 
potential  energy  based  on  the  maximum  observed  height  was 
between  246  /iJ  and  262  /iJ. 

The  data  points  were  then  fitted  to  the  parabolic  curve 
describing  the  motion  of  a  projectile  under  constant  free-fall 
acceleration  g  and  negligible  air  resistance.  The  parabolic  curve 


Fig.  8.  Propelled  hexapod  when  actuator  triggered  by  hght  captured  at  1000 
frames  per  second.  Frames  shown  above  are  20  ms  apart.  The  vertical  scale  bar 
shown  is  spaced  1  cm  per  division. 
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Fig.  9.  Trajectory  of  the  hexapod  jump.  Experimentally  determined  height 
shown  as  blue  circles.  The  parabolic  curve  fit  describes  projectile  motion  under 
acceleration  due  to  gravity  and  negligible  air  resistance. 


was  then  fitted  to  the  data  to  determine  the  initial  veloc- 
ity Vi,  as  shown  in  Fig.  9.  This  fit  resulted  in  a  calculated 
take-off  velocity  of  1.263  m/s.  The  associated  kinetic  energy 
is  250  /iJ,  which  is  within  the  potential  energy  uncertainty 
bounds  calculated  based  on  measured  maximum  height.  The 
quality  of  the  fit  to  the  experimental  data  also  demonstrates 
that  the  microrobotic  platform  followed  a  drag-free  parabolic 
trajectory  under  acceleration  due  to  gravity  and  negligible 
air  resistance. 

Although  the  overall  mass  of  the  energetic  chip  as  given  in 
Table  I  was  approximately  25  mg,  the  mass  of  the  reactive 
region  was  estimated  at  only  0.21  mg.  The  2  mm  diameter 
energetic  region  was  able  to  propel  the  robot  approximately 
8  cm  into  the  air,  even  though  the  robot  was  more  than 
1000  times  the  mass  of  the  energetic  material. 

As  a  point  of  comparison,  to  achieve  the  same  height  with 
a  conventional  actuator  would  require  an  actuator  stiffness 
of  2.4  kN/m  and  a  force  of  at  least  9.6  N  (assuming  the 
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leg  length  of  4  mm  is  the  actuator  throw).  This  would  be 
very  difficult  if  not  impossible  to  achieve  at  this  size  scale. 
To  ignite  the  nanoporous  energetic  silicon,  a  maximum  of 
180  /iJ  of  electrical  energy  was  consumed  (assuming  that  the 
10  /iF  capacitor  discharged  entirely  from  6  V  to  0  V).  Defining 
an  "effective  actuation  efficiency"  as  the  ratio  of  mechanical 
energy  produced  to  electrical  energy  consumed,  the  8  cm  jump 
here  represented  an  effective  efficiency  of  137%.  Based  on 
past  measurements  of  the  energy  density  of  the  nanoporous 
energetic  silicon  reaction,  approximately  1.9  J  of  chemical 
energy  was  released  [21].  Therefore,  the  chemical  to  kinetic 
conversion  efficiency  was  actually  extremely  low  (0.0127%), 
indicating  that  the  vast  majority  of  the  available  energy  was 
wasted. 

There  is  consequently  room  for  improvement,  and  the  au- 
thors believe  implementation  of  a  nozzle  in  future  designs  will 
increase  this  efficiency  by  directing  gases  in  a  more  effec- 
tive manner  to  produce  maximum  thrust.  However,  the  fact 
that  a  respectable  height  of  over  11  x  the  body  length  was 
achieved  despite  the  abysmal  conversion  efficiency  underscores 
the  tremendous  amount  of  energy  available  in  the  nanoporous 
energetic  silicon.  The  authors  believe  that  this  technology  will 
have  many  applications  in  future  MEMS  systems  if  the  energy 
can  be  more  efficiently  captured  and  put  to  use. 


V.  Conclusion 

An  autonomous  jumping  microrobot  has  been  demonstrated, 
achieving  a  vertical  jump  of  8  cm,  which  was  over  1 1  times  its 
longest  physical  dimension.  Autonomy  was  achieved  through 
on-board  integration  of  a  simple  circuit  constructed  of  dis- 
crete surface  mount  components.  The  resulting  system  has 
optical  sensing,  control,  power,  and  mobility.  The  rudimen- 
tary sensing  and  intelligence  allow  the  hexapod  to  detect  an 
increase  in  light  intensity  and  respond  by  jumping.  In  ad- 
dition, the  novel  use  of  an  energetic  material  for  actuation 
was  shown,  allowing  significantly  more  mechanical  energy  to 
be  produced  than  the  electrical  energy  needed  to  initiate  the 
reaction. 

A  hybrid  integration  approach  has  also  been  presented,  which 
uses  a  low-temperature  solder  and  metal  interconnects  pat- 
terned directly  on  the  robot's  polymer  chassis  to  integrate  var- 
ious discrete  electrical  components.  One  of  the  unique  aspects 
to  the  integration  approach  is  that  a  single  UV-curable  polymer 
material  was  used  for  the  robot  chassis,  as  a  substrate  for  the 
assembly  of  the  circuit  components,  and  as  a  packaging  mate- 
rial. The  authors  expect  improvements  in  direction  control  and 
development  of  an  array  of  energetic  actuators  to  allow  multiple 
jumps,  leading  to  future  technological  leaps  for  autonomous 
microrobots. 
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This  article  presents  a  Kalman  filter  based  adaptive  disturbance  accommodating  stochastic  control  scheme  for 
linear  uncertain  systems  to  minimise  the  adverse  effects  of  both  model  uncertainties  and  external  disturbances. 
Instead  of  dealing  with  system  uncertainties  and  external  disturbances  separately,  the  disturbance  accommodat- 
ing control  scheme  lumps  the  overall  effects  of  these  errors  in  a  to-be-determined  model-error  vector  and  then 
utilises  a  Kalman  filter  in  the  feedback  loop  for  simultaneously  estimating  the  system  states  and  the  model-error 
vector  from  noisy  measurements.  Since  the  model-error  dynamics  is  unknown,  the  process  noise  covariance 
associated  with  the  model-error  dynamics  is  used  to  empirically  tune  the  Kalman  filter  to  yield  accurate  estimates. 
A  rigorous  stochastic  stabiHty  analysis  reveals  a  lower  bound  requirement  on  the  assumed  system  process  noise 
covariance  to  ensure  the  stabiHty  of  the  controlled  system  when  the  nominal  control  action  on  the  true  plant  is 
unstable.  An  adaptive  law  is  synthesised  for  the  selection  of  stabilising  system  process  noise  covariance. 
Simulation  results  are  presented  where  the  proposed  control  scheme  is  implemented  on  a  two  degree-of-freedom 
helicopter. 

Keywords:  disturbance  accommodating  control;  stochastic  adaptive  control;  Kalman  filter;  stochastic  stabiHty 


1.  Introduction 

Uncertainty  in  dynamic  systems  may  take  numerous 
forms,  but  among  them  the  most  significant  are  noise/ 
disturbance  uncertainty  and  model/parameter  uncer- 
tainty. External  disturbances  and  system  uncertainties 
can  obscure  the  development  of  a  viable  control  law. 
This  article  presents  the  formulation  and  analysis  of 
a  stochastic  robust  control  scheme  know^n  as  the 
Disturbance  Accommodating  Control  (DAC).  The 
main  objective  of  DAC  is  to  make  necessary  correc- 
tions to  the  nominal  control  input  to  accommodate  for 
external  disturbances  and  system  uncertainties.  Instead 
of  deahng  with  system  uncertainties  and  disturbances 
separately,  DAC  lumps  the  overall  effects  of  these 
errors  in  a  to-be-determined  term  that  is  used  to 
directly  update  a  nominal  control  input. 

DAC  was  first  proposed  by  Johnson  in  1971 
(Johnson  1971).  Though  the  traditional  DAC 
approach  only  considers  disturbance  functions  which 
exhibit  waveform  patterns  over  short  intervals  of  time 
(Johnson  and  Kelly  1981),  a  more  general  formulation 
of  DAC  can  accommodate  the  simultaneous  presence 
of  both  'noise'  type  disturbances  and  'waveform 
structured'  disturbances  (Johnson  1984,  1985).  The 
disturbance  accommodating  observer  approach  has 
been  shown  to  be  extremely  effective  for  disturbance 
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attenuation  (Profeta,  Vogt,  and  Mickle  1990;  Kim  and 
Oh  1998;  Biglari  and  Mobasher  2000);  however,  the 
performance  of  the  observer  can  significantly  vary  for 
different  types  of  exogenous  disturbances,  which  is  due 
to  observer  gain  sensitivity. 

This  article  presents  a  robust  control  approach 
based  on  a  significant  extension  of  the  conventional 
observer-based  DAC  concept,  which  compensates  for 
both  unknown  model  parameter  uncertainties  and 
external  disturbances  by  estimating  a  model-error 
vector  (throughout  this  article  the  phrase  'disturbance 
term'  will  be  used  to  refer  to  this  quantity)  in  real  time. 
The  estimated  model-error  vector  is  further  used  as  a 
signal  synthesis  adaptive  correction  to  the  nominal 
control  input  to  achieve  maximum  performance.  This 
control  approach  utilises  a  Kalman  filter  in  the 
feedback  loop  for  simultaneously  estimating  the 
system  states  and  the  disturbance  term  from  measure- 
ments (Sorrells  1982,  1989;  Davari  and  Chandramohan 
2003).  The  estimated  states  are  then  used  to  develop  a 
nominal  control  law  while  the  estimated  disturbance 
term  is  used  to  make  necessary  corrections  to  the 
nominal  control  input  to  minimise  the  effects  of  both 
system  uncertainties  and  the  external  disturbance. 
There  are  several  advantages  of  implementing  the 
Kalman  filter  in  the  DAC  approach:  (i)  tuning  of  the 
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estimator  parameters,  such  as  the  process  noise  matrix, 
can  be  done  easily  unhke  conventional  DAC 
approaches  in  which  the  adaptation  involves  the 
entire  feedback  gain;  (ii)  the  estimated  disturbance 
term  is  a  natural  byproduct  of  state  estimation  and 
(iii)  the  Kalman  filter  can  be  used  to  filter  noisy 
measurements. 

It  is  a  well-known  fact  that  the  closed-loop 
performance  and  the  stability  of  the  Kalman  filter- 
based  DAC  approach  depends  on  the  accuracy  of  the 
estimated  disturbance  term.  Since  the  dynamics  of  the 
disturbance  term  is  unknown,  the  process  noise 
covariance  associated  with  the  disturbance  term  is 
used  to  empirically  tune  the  Kalman  filter  to  yield 
accurate  estimates.  Although  the  Kalman  filter-based 
DAC  approach  has  been  successfully  utiHsed  for 
practical  appHcations,  there  has  not  been  any  rigorous 
stochastic  stabihty  analysis  to  reveal  the  interdepen- 
dency  between  the  estimator  process  noise  covariance 
and  controlled  system  stability.  The  first  main  con- 
tribution of  this  article  is  a  detailed  stabihty  analysis, 
which  examines  the  expHcit  dependency  of  the  con- 
trolled system's  closed-loop  stabihty  on  the  disturbance 
term  process  noise  covariance  and  the  measurement 
noise  covariance.  Since  the  system  under  consideration 
is  stochastic  in  nature,  the  notion  of  stability  is  depicted 
in  two  separate  fashions.  The  first  method  deals  with 
moment  stabihty  and  the  second  technique  considers 
stabihty  in  a  probabihstic  sense. 

Stochastic  stabihty  analysis  on  the  Kalman  filter- 
based  DAC  approach  indicates  that  the  effectiveness  of 
the  proposed  control  scheme  depends  on  the  estimator 
parameters  such  as  the  process  noise  covariance 
matrix.  The  stabihty  analysis  also  indicates  that  the 
DAC  scheme  is  most  effective  when  the  assumed 
process  noise  covariance  satisfies  a  lower  bound 
requirement  which  depends  on  the  system  uncertain- 
ties. In  general,  it  is  difficult  to  select  a  stabihsing 
process  noise  covariance  for  the  broad  type  of 
uncertain  systems  considered  here.  One  could  always 
try  to  select  an  extremely  large  value  of  process  noise 
covariance  that  might  stabihse  the  system  or  even 
monotonically  increase  the  process  noise  covariance 
matrix  in  an  ad-hoc  manner  until  the  system  stabihses. 
However,  it  is  important  to  keep  in  mind  that  selecting 
a  large  process  noise  covariance  matrix  would  result  in 
noisy  control  signal  which  could  lead  to  problems  such 
as  chattering.  The  second  main  contribution  of  this 
article  is  the  formulation  of  a  stochastic  adaptive 
scheme  for  selecting  the  appropriate  process  noise 
covariance  that  would  guarantee  closed-loop  stability 
of  the  controlled  system. 

The  structure  of  this  article  is  as  follows.  A  detailed 
formulation  of  the  stochastic  DAC  approach  for  multi- 
input  multi-output  (MIMO)  systems,  followed  by  a 


stochastic  stabihty  analysis,  is  first  given.  Next,  an 
adaptive  scheme  is  developed  for  the  selection  of 
stabilising  the  disturbance  term  process  noise  covar- 
iance. Simulation  results  are  then  presented  where  the 
proposed  control  scheme  is  implemented  on  a  two 
degree-of-freedom  helicopter. 


2.  DAC  formulation 

Let  (Q,  P)  denote  a  probability  space,  where  ^  is  the 
sample  space,  ^  is  a  a-field  and  P  is  a  probability 
measure  on  the  measurable  space  J^).  Additionally, 
the  elements  of  ^2  are  denoted  by  co  and  the  members 
of  T  are  called  events.  Now  consider  a  hnear  time- 
invariant  stochastic  system  of  the  following  form: 

Xi(0  =  iiXi(0  +  ^2X2(0,  Xi(^)  =  Xi„, 

X2(0  =  ^3X1(0  +  ^4X2(0  +  Bvit)  +  W(0,  (1) 

X2(^)  =  X20. 

Here,  the  stochastic  state  vector,  [Xf(0  X^(0]^= 
X(0  =  X(^,  oS)  :  [?o,  r/]  X  ^  is  an  77-dimensional 

random  variable  for  fixed  t.  The  state  vectors,  Xi(0 
and  X2(0  are  of  dimensions  Xi(0  =  Xi(/,  cS) :  [/q,  tf  \  x 
^  m"-'  and  X2(0  =  X2(r,  co) :  [to,  tj]  x  Q  ^  m\ 
respectively.  The  system  given  in  (1)  is  in  the  typical 
kinematics-dynamics  form,  where  the  kinematics  is 
assumed  to  be  fully  known,  i.e.  the  state  matrices 
^^^^(n-r)Mn-r)  ^2  ^  ^^""'"^"^    are  prcciscly 

known.  Uncertainty  is  only  associated  with  the 
dynamics,  i.e.  the  state  and  control  distribution 
matrices,  ^3  e  ^'""^""'"^  A^^W\  B^W\  are 
assumed  to  be  unknown.  Also,  the  input  matrix  B  is 
assumed  to  be  non-singular.  Finally,  the  stochastic 
external  disturbance  W(0  =  W(/,  (d)  :  [/q,  tf\xQ.^ 
is  modelled  as  a  hnear  time-invariant  system  driven  by 
a  Gaussian  white  noise  process,  i.e. 

W(0  =  L(X(0,  W(0)  +  V(0,    W(^)  =  0,xi,  (2) 

where  [L()  is  an  unknown  linear  operator  and 
V(0  =  V{t,  o)) :  tf\xQ.^  is  assumed  to  be 
zero-mean  Gaussian  white  noise  process,  i.e.  V(0"^ 
A/'(0,  Q<5(t)).  It  is  important  to  note  that  the  hnear 
operator  IL(-)  and  the  covariance  of  the  white  noise 
process  V(0?  are  unknown.  The  measurement  equation 
is  given  as 

Y(0  =  CX(0  +  V(0,  (3) 

where  Y(0  =  Y(?,  cd)  :  [?o,  x  ^  Oi"^  is  the  measure- 
ment vector  and  C^W^^^  denotes  the  known  output 
matrix.  The  measurement  noise,  V(0  =  V(/,  cd)  : 
X  ^  D^t'^,  is  assumed  to  be  zero-mean 
Gaussian  white  noise  with  known  covariance,  i.e. 
V(0-^A/'(0,i?5(r)). 


The  assumed  (known)  system  matrices  are  given  as 
A^^,  and  B^^.  Now  the  external  disturbance  and  the 
model  uncertainties  can  be  lumped  into  a  disturbance 
term,  T>(t)  G  W,  through 

T>(t)  =  AAiXi(t)  +  A^2X2(0  +  A5u(0  +  W(0,  (4) 

where  AAi  =  (A^  —  A^^),  AA2  =  (A4  —  A^J  and 
AB  —  (B  —  Bfn)'  Using  this  disturbance  term  the  true 
model  can  be  written  in  terms  of  the  known  system 
matrices  as  follows: 


Xi(0  =  ^1X1(0  +  ^2X2(0, 

X2(0  =  ^3.Xi(0  +  ^4^X2(0  +  B^u(t)  +  x>(0. 


(5) 


The  control  law,  u(0,  consists  of  a  nominal  control  and 
a  control  correction  term  to  minimise  the  adverse  effect 
of  the  disturbance  term,  1^(0,  i-e. 


u(0  =  u(0  +  u(0. 


(6) 


Here  u(0  is  the  nominal  control  and  u(0  is  the  control 
correction  term.  For  the  purpose  of  analysis,  the 
control  correction  term  is  selected  to  ensure  the 
complete  cancellation  of  the  disturbance  term.  Thus 
the  DAC  law  can  be  written  as 


u(0  =  0(0-^-^^(0. 


(7) 


The  disturbance  term  is  not  known,  but  an  estimator 
can  be  implemented  in  the  feedback  loop  to  estimate 
the  disturbance  term  online.  Estimating  the  distur- 
bance term  requires  knowledge  of  its  dynamic  model. 
Since  the  dynamics  of  the  disturbance  term  is  not 
precisely  known,  the  disturbance  term  dynamics  is 
modelled  as 

-Dmit)  =  AvMO  +  >V(0,  =  0,  (8) 

where  Ap^  is  Hurwitz  and  W(0  =  W(/,  co) :  [^o,  t/]  x 
Q  ^  is  zero-mean  Gaussian  white  noise  process, 
i.e.  W(0~A/'(0,  Q8(t)).  Equation  (8)  is  used  solely  in 
the  estimator  design  to  estimate  the  true  disturbance 
term.  After  constructing  the  assumed  augmented  state 

vector  as  Z^(0  =  [Xf^(0  X2^(0  X>^(0]^,  the  assumed 
extended  model  of  the  system  can  be  written  as 


"Xi„(0" 

X2„(0 

.i>m(0. 

Ai  A2 

Aim  A4,,, 

^rx(n—r)  ^rxr 
r)xr 


0, 


\n—r)xr 
Ij-xr 


■Xu(0' 
X2JO 
.x>„,(0 


u(0  + 


0(n-r)xl 
0.x  1 


(9) 


The  zero  elements  in  the  disturbance  term  dynamics 
are  assumed  for  the  sake  of  simphcity;  however,  the 
control  formulation  given  here  is  also  valid  if  non-zero 


elements  are  assumed.  Equation  (9)  can  be  written  in 
terms  of  the  appended  state  vector,  Z^,  as 


Z^(0  =  F^Zm(t)  +  /)^u(0  +  GW(t), 


(10) 


where 


A2 

r)xr 

r)xr 

A3,,. 

Aa„ 

Jrxr 

,    D,„  = 

^rx(n—r) 

Orxr  _ 

Irxr  A 


Note  that  the  uncertainty  is  only  associated  with  the 
dynamics  of  the  disturbance  term.  Let  Z(0  = 
[Xf(0  ^lit)  X>^(0]^  and  H=[C  O^xr]-  Now  the 
measured  output  equation  can  be  written  as 


Y(0  =  //z(0  +  v(0. 


(11) 


Though  the  disturbance  term  is  unknown,  an  estimator 
such  as  a  Kalman  filter  can  be  implemented  in  the 
feedback  loop  to  estimate  the  unmeasured  system 
states  and  the  disturbance  term  from  the  noisy 
measurements.  Let  Z(0  =  [Xf(0  Xf(0  i>^(0]^,  now 
the  estimator  dynamics  can  be  written  as 


Z(0  =  F^Z{t)  +  DmKt)  +  ^(0[Y(0  -  Y(0], 

z(^)  = 


(12) 


where  K{t)  is  the  Kalman  gain  and  Y(0  =  HZ{t).  The 
Kalman  gain  can  be  calculated  as  K(t)  =  P(t)H^R~^, 
where  P(t)  is  obtained  by  solving  the  continuous-time 
matrix  differential  Riccati  equation  (Crassidis  and 
Junkins  2004): 


-  P{i)H^R-^HP{i)  +  GQG^, 


Pito).  (13) 


Let  Z(0  =  [Xf(0  Xf(0  X>^(0]^,  now  the  estimator 
dynamics  can  be  rewritten  as 

z(o = F^z{t) + D^t) + mmm  -  m] + ^(ov(o. 

(14) 

The  estimator  uses  the  assumed  system  model  given 
in  (10)  for  the  propagation  stage  and  the  actual 
measurements  for  the  update  stage,  i.e.  Z(0  = 
E[Z^(0|{Y(^o) . . .  Y(0}].  Due  to  system  uncertainties, 
the  estimator  in  (14)  is  sub-optimal  and  the  estimates 
Z(0  may  be  biased. 

Remark  1:  Accuracy  of  the  estimates  depends  on  Q 
which  indicates  how  well  the  disturbance  term 
dynamics  is  modelled  via  (8).  A  large  Q  indicates 


that  (8)  is  a  poor  model  of  the  true  disturbance  term 
dynamics  and  a  small  Q  indicates  that  (8)  is  an  accurate 
model  of  the  true  disturbance  term  dynamics.  Note 
that  selecting  a  small  Q,  while  having  a  poor  model, 
would  result  in  inaccurate  estimates. 

The  total  control  law,  u(0,  consists  of  a  nominal 
control  and  necessary  corrections  to  the  nominal 
control  to  compensate  for  the  disturbance  term  as 
shown  in  (7).  The  nominal  control  is  assumed  to  be  a 
state  feedback  control,  where  the  feedback  gain, 

Km  =  \Km^    Km^,   is   SClcCtcd   SO   that   {Am  —  BmKrr)  is 

Hurwitz,  where 


Am  — 


^3. 


Aa^ 


and    Bm  = 


J 


While  the  nominal  controller  guarantees  the  desired 
performance  of  the  assumed  model,  the  second  term, 
— X>(0,  in  (7)  ensures  the  complete  cancellation  of  the 
disturbance  term  which  is  compensating  for  the 
external  disturbance  and  model  uncertainties.  Now 
the  DAC  law  can  be  written  in  terms  of  the  estimated 
system  states  and  the  estimated  disturbance  term  as 


X(t) 
J>(t) 


=  SZ(t),  (15) 


where  S=  —[Km  B'^].  Note  that  B^  is  assumed  to  be  a 
non-singular  matrix.  A  summary  of  the  proposed 
control  scheme  is  given  in  Table  1. 

Remark  2:  It  is  important  to  note  that  if  2^0,  then 
^m(0  ^  ^m(^)  =  0  and  the  total  control  law  given  in 
(15)  becomes  just  the  nominal  control.  If  the  nominal 
control,  u(0,  on  the  true  plant  would  result  in  an 
unstable  system,  i.e.  the  matrix  {A  —  BK^^  is  unstable, 
where 


A  = 


Ai  A2 
A3  A4 


and    B  = 


^(n—r)xr 

B 


then  selecting  a  small  Q  would  also  result  in  an 
unstable  system.  On  the  other  hand,  selecting  a  large  Q 
value  would  compel  the  estimator  to  completely  rely 
upon  the  measurement  signal  and  therefore  the  noise 
associated  with  the  measurement  signal  is  directly 
transmitted  into  the  estimates.  This  could  result  in  a 
noisy  control  signal  which  could  lead  to  problems  such 
as  chattering.  Also  note  that  as  R,  the  measurement 
noise  covariance,  increases,  the  estimator  gain 
decreases  and  thus  the  estimator  fails  to  update  the 
propagated  disturbance  term  based  on  measurements. 
Thus,  for  a  highly  uncertain  system,  if  the  nominal 
control  action  on  the  true  plant  results  in  an  unstable 
system,  then  selecting  a  small  Q  or  a  large  R  would  also 
result  in  an  unstable  closed-loop  system. 

If  the  estimator  in  (14)  is  able  to  obtain  accurate 
estimates  of  the  system  states  and  the  disturbance  term, 
then  the  control  law  in  (15)  guarantees  the  desired 
closed-loop  performance.  The  accuracy  of  the  esti- 
mated system  states  and  the  disturbance  term  depends 
on  the  estimator  parameters  such  as  the  process  noise 
covariance,  g,  and  the  measurement  noise  covariance, 
R.  Thus  the  performance  of  the  DAC  approach 
presented  here  depends  on  the  estimator  design 
parameters.  A  schematic  representation  of  the  pro- 
posed controller  is  given  in  Figure  1 . 


3.  Stochastic  stabiKty  analysis 

Without  the  loss  of  generahty,  the  following  assump- 
tion can  be  made  about  the  external  disturbance 
model. 

Assumption  3.1:    The  linear  operator,  [L(-),  in  the 

external  disturbance  term  model  in  (2)  is  assumed  to  be 


L(X(o,  w(0)  =  ^wiXi(0  +  ^w.X2(0  +  w(0, 


(16) 


where  A^^ ,  Ay,^  and  Ay^^  are  unknown  matrices. 


Table  1.  Summary  of  DAC  control  process. 


Plant  Xi(0  =  ^1X1  (0  +  ^2X2(0 

X2(0  =  ^3.Xi(0  +       X2(0  +  B^u{t)  +  x>(0 
Y(0  =  CX(0  +  V(0 
Initialise  Z(?o),  ^'(^o) 

Estimator  gain  P{t)  =  F^P(t)  +  P(t)F^  -  P{t)H^ R-^HP{t)  +  GQG^ 

K(t)  =  P(t)H^R-^ 

Estimate  Z(0  =  iVzZ(0  +  D^\\{t)  +  K{t)\Y{t)  -  Y(0] 

Control  synthesis  u(0  =  -[Km  B-^]Z(t) 
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Based  on  Equation  (4),  the  true  disturbance  term 
dynamics  can  now  be  written  as 

T>(t)  =  AAiXiit)  +  A^2X2(/)  +  A^u(0 

+  L(X(0,W(0)  +  V(0.  (17) 

Substituting  the  control  law  (15)  the  above  equation 
can  be  written  as 

T>(t)  =  AAiXi(t)  +  A^2X2(0  +  ABSZ(t) 
+  L(X(0,W(0)  +  V(0 
=  A^i{iiXi(0  +  i2X2(0} 
+  A^2{^3.Xi(0  +  ^4^X2(0  +  B^SZ(t)  +  X>(0} 
+  ABS{[F^  +  D^S  -  K(t)H] 

X  Z(0  +  K{t)HZ{t)  +  K{i)\{t)} 

+  ^wiXi(0  +  ^w,X2(0  +  ^W3W(0  +  V(0. 

Assume  that  the  output  matrix  can  be  partitioned  as 
C  =  [Ci  C2],  and  H  can  be  written  as  H  = 
[Ci  C2  O^xr].  Thus  K(t)HZ(t)  =  K(t)CiXi(t)-\- 
K{i)C^2{t).  Also  note  that  W(0  can  be  written  as 
W(0  =  X>(0-A^iXi(0-A^2X2(0-A5u(0-  Now  the 
true  disturbance  term  dynamics  can  be  written  as 

Tf{i)  =  {AAxAi  +  A^2^3.  +  ABSK{t)Cx 

+  ^Wi  -^W3A^l}Xl(0 

+  [AAxA2  +  A^2^4^  +  ABSK(t)C2 

+  -^W3A^2}X2(0 

+  {ABS[F^  +  D^S  -  K(t)H] 

+  AA2BmS  -  Ay,^ABS}Z{t) 

+  { A^2  +      }^(0  +  ABSK{t)\{t)  +  V(0. 

Let  ABSK(t)Y(t)  +  V(0  =  Wa(0,  thus  Wa(t)  is  also  a 
zero-mean  stochastic  process  with 

E[Wa(t)Wl(t  +  T)]  =  {ABSK(t)RK^(t)S^AB^+Q]8(T) 

=  Qa(t)8(T). 


Note  that  T>(t)  is  a  linear  function  of  the  true  extended 
system  state,  Z(t),  the  estimated  states,  Z(t),  and  the 
noise  term,  Wa(0-  Thus  the  system  in  (1)  is  rewritten  as 
the  following  extended  dynamically  equivalent  system: 


Al  A2  0(n-r)xr 

^rxr 

AvXt)    Av,{t)  Av,{t) 


"Xi(0" 

X2(0 

"Xi(/)- 

X2(0 

.  ^(0  _ 

^{n—r)x{n+r) 

Bf^S 

z(0  + 

Orxl 

Bv(t) 

_  Wait)  _ 

where 


AvXt)  =  {AAiAi  +  A^2^3^  +  ABSK{i)Cx 


^W3A^lj 


Av^{t)  =  {A^ii2  +  A^2^4^  +  ABSK(t)C2 
+  ^W2  -Ay,^AA2}, 

Av,(t)  =  {AA2 A^^} 


and 


Bv(t)  =  {ABS[F^  +  D^S  -  K(t)H] 
^AA2B^S-A,,^ABS}. 

Equation  (18)  can  be  written  in  concise  form  as 


Z(0  =  F(t)Z(t)  +  D(t)Z(t)  +  GWaitX 


where 


F(t)  = 


Al  A2  0(ji-r)xr 

A4^  Irxr 

Av,(t)    Av,{i)  Av,{t)  . 

0(n-r)x(n+r) 
BmS 

Bv{i) 


and 


(18) 


(19) 


Ref.  signal 


X(t) 


|W(t) 


Nominal  controller 


u(^),  0  u(^) 


Plant 


|V(t) 


Estimator 


Figure  1 .  DAC  block  diagram. 
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After  substituting  the  control  law,  the  estimator 
dynamics  can  be  written  as 

Z(0  =  F^Z(0  +  DmSZ{t)  +  K(t)H[Z(t)  -  Z(t)] 

Let  Z(t)  =  Z(t)  —  Z{t)  be  the  estimation  error,  then  the 
error  dynamics  can  be  written  as 

z(o  =  [Fm  -  mH  +  AF(0]Z(0  +  [^F{t)  +  ^D{t)\z{t) 

+  GW«(0-^(OV(0,  (20) 

where  ^F{t)  =  F{t)  -  F^  and  A  D{t)  D{t)  -  D^S. 
Combining  the  error  dynamics  and  the  estimator 
dynamics  yields 


m 

i(0 


(F„  -  K(t)H  +  AF(0)    (AF(0  +  AD(t))  1 
K(t)H  (F^  +  D,„S)  J 


+ 

[z(o_ 

G  -m-\im(tr 

0(«+r)xr  ^(0 


T4n 


or  in  a  more  compact  form  as 

z(t)  =  r{t)Z(t)  +  T{t)g{t), 


(21) 
(22) 


where 


T(/) 


r(0: 


Z{t): 


(F^-mH+AFit)) 
K{t)H 
G  -Kit) 

0(«+r)xr  -^^(0 

rwa(o 

L   V(0  J 


(AF{t)  +  AD{t)) 
{F„,+D„,S) 


r  G 

LO(«+r)xr       -^^(0  J' 

[!;!]• « 


Although  the  Kalman  filter-based  DAC  approach 
has  been  successfully  utihsed  for  practical  applications, 
there  has  not  been  any  rigorous  stochastic  stabihty 
analysis  to  reveal  the  interdependency  between  the 
estimator  process  noise  covariance  and  controlled 
system  stabihty.  Since  the  system  under  consideration 
is  stochastic  in  nature,  the  notion  of  stability  is 
depicted  in  two  separate  fashions.  The  first  method 
deals  with  moment  stabihty;  for  the  Gaussian  stochas- 
tic processes  presented  here,  the  first  two  moments  are 
considered.  The  second  technique  considers  stability  in 
a  probabiHstic  sense. 


3.1  First  moment  stability 

In  this  section  a  detailed  stability  analysis  which 
examines  the  explicit  dependency  of  the  controlled 
system's  first  moment  stabihty  or  the  mean  stabihty  on 
the  estimator  parameters,  such  as  the  disturbance  term 


process  noise  covariance  Q  and  the  measurement  noise 
covariance  R,  is  given.  First,  a  few  definitions 
regarding  the  closed-loop  system's  mean  stabihty  are 
given.  These  definitions  and  notations  are  first 
introduced  for  a  system  without  any  parameter 
uncertainties  and  are  used  throughout  the  rest  of  this 
article. 


3.1.1  System  without  uncertainties 
Here  a  system  without  any  parameter  uncertainties 
is  considered,  i.e.  F(t)  =  F^,  D{t)  =  D^S,  and 
W^(/)  =  W(0-  If  there  is  no  model  error,  then  the 
estimator  is  unbiased,  i.e.  ^[Z(0]  =  ^^{t)  =  0.  Note 
that  the  overline  is  used  to  indicate  the  states  of  the 
system  when  there  is  no  model  uncertainties.  Now  (21) 
may  be  written  as 


z(0 

i(o. 


+r)x(n+r)  "I 


Z(0 
Z(0 


W(0] 
v(0  J' 


Fm  —  K(t)H  0(^n+r)x 
.       mH  Fm 

G  -K{ty 

^(n+r)xr       K{t)  _ 

where  Z(t)  and  Z(t)  denote  the  estimation  error  and 
estimated  states  when  there  is_  no  model  error, 
respectively.  Let  Z(t)  =  [Z^(t)  Z\t)Y  and  g(t)  = 
[W^(0  V^(0]^,  now  the  above  equation  can  be 
written  in  a  more  compact  form  as 

k(t)  =  T(t)Z(t)  +  (23) 

where 

Fm  —  K{t)H  ^(n+r)x{n+r) 

K(t)H       Fm  +  DmS_ 

Note  that  Q{t)  is  a  zero-mean  Gaussian  white  noise 
process  with 


t(0 


E[g(t)G\t  -  r)]  : 


r  Q  o,x. 

\_Omxr  R 


8(r)  =  A8(r). 


Since  the  first  moment  stability  is  of  concern  here, 
the  first  moment  dynamics  or  the  mean  dynamics  is 
written  as 


E[Z(t)]  =  ii^(t)  =  r(t)fi^(ty 


(24) 


Definition  3.2:  Given  M  >  1  and  p  edi,  the  system  in 
(23)  is  said  to  be  (M,  )6)-stable  in  the  mean  if 

m,  to)fi^(to)\  <  Me^^'-'^^\^^(to)\    "it  >  ^,  (25) 

where  t^)  is  the  evolution  operator  generated  by 
T(/)  and  \-\  indicates  the  Euclidean  norm,  i.e. 


|m|  = 


'2  +      +  • 
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Since  most  applications  involve  the  case  where 
<  0,  (M,  yS)-stabiHty  guarantees  both  a  specific  decay 
rate  of  the  mean  response  (given  by  p)  and  a  specific 
bound  on  the  transient  behaviour  of  the  mean 
(given  by  M). 

Definition  3.3:  If  the  stochastic  system  in  (23)  is 
(M,  y0)-stable  in  the  mean,  then  the  transient  bound  of 
the  system  mean  response  for  the  exponential  rate  p  is 
defined  to  be 

=  infjM  G  9ft;     >  ^0  :  II ^(^,  ^)ll  <  Me^^'-''^Y 


(26) 


Here 


indicates  the  matrix  two-norm,  i.e. 


IIMII 


x(M), 


where  (Jmaxi')  denotes  the  maximum  singular  value. 

As  shown  in  Theorem  3.4,  the  (M,  y6)-stabihty  and 
the  transient  bound  of  the  system's  mean  response  are 
related  to  a  continuous  time  Lyapunov  matrix 
differential  equation. 

Theorem  3.4:  Assume  that  there  exists  a  bounded, 
continuously  differ entiable  positive  definite  matrix  func- 
tion V{i)  satisfying  the  Lyapunov  matrix  differential 
equation 

P(t)  =  r(t)V(t)  +  v(t)r\t)  +  r(OAr^(o,  V(to) 

(27) 

then  the  system  in  (23)  is  (M,  p)-stable  in  the  mean  and 
the  transient  bound  of  the  system  mean  response  can 
be  obtained  as 


Now  (28)  follows  from 

cTmUV(t))/a^Unto))  >  l|0(/,/o)ll' 


t  >  to. 


□ 


Remark  3:_  Assume    that    V(to)  is    selected  as 

V{to)  =  E[Z{to)  Z^{to)l  then  the  positive  definite 

solution,  V{t),  satisfying  Equation  (27)  denotes  the 
correlation  matrix,  i.e. 

V(t)  =  E[Z(t)  Z\t)l 

Thus  the  transient  bound  of  the  system  mean  response 
can  be  obtained  in  terms  of  the  bounded  correlation 
matrix. 

Note  that  r(OAr^(0  in  (27)  can  be  factored  as 
shown  below: 


r(OAr^(0  = 


{GQG^-\-KRK^  -KRK^' 
-KRK^  KRK^ 


GQG^  -0" 
0       0  . 

G 
0 


+ 


KRK' 
-KRK^ 


-KRK' 
KRK^ 


=  LQL^-\-N(t)R-^N^(t), 


where 


[:] 


and    N(t)  ■■ 


r  pm^  ] 


Thus  (27)  can  be  written  as 


V(t)  =  r(t)V(t)  +  V(t)r\t)  +  LQL^  +  N(t)R-^N\t). 

(29) 


Mi  <  sup  CTrr,WP(t))/crmm(P(to)l 

t>to 


(28) 


where  crmin(-)  denotes  the  minimum  singular  value. 

Proof:  Since  r(OAr^(0  >  0  >  ^,  the  (M,P)- 
stabihty  in  the  mean  follows  directly  from  the  existence 
of  bounded  positive  definite  solution,  V(t),  satisfying 
Equation  (27).  Now  the  solution  to  (27)  can  be 
written  as 

V(t)  =  6(^,  to)V(to)^\t,  to) 


+ 


J  to 


r)r(T)Ar^(T)0^(^,  r)dr 


Note_  that  Vt  ^  to,  V{t)  >  <^(t,  to)V(to)^^(t,  to)  > 
crmin(V(to))Ht,  to)^^(t,  to),  i.e. 

cTmunt))  >  \m,to)v(to)^\t,to)\\ 

>CTrnUHt0))\\Ht.t0)\\\      t  >  to. 


3.1.2  Uncertain  system 

In  this  section,  the  first  moment  stability  of  the 
perturbed  system  given  in  (22)  is  considered,  i.e. 

z{t)  =  r(t)Z(t)  +  Ar(t)Z(t)  +  r(t)g(t),  (30) 

where 

-AF(t)    (AF(t)  ^  AD(t))l 
0  0  J* 


AT(0  = 


The  correlation  matrix  V(t)  =  E[Z(t)  Z^(t)]  satisfies 
the  following  matrix  Lyapunov  differential  equation: 


p(t)  =  (t(/)  +  Ar(t))V(t)  +  V(t){f(t)  +  AT(0) 

+  r(OA(or^(0, 

where 


(31) 


Assuming  that  the  nominal  control  action  on  the  true 
plant  would  result  in  an  unstable  system,  stabihty  of 
extended  uncertain  system  given  in  (30)  depends  on  the 
disturbance  term  process  noise  covariance,  Q  and 
the  measurement  noise  covariance,  R.  Theorem  3.5 
indicates  that  the  stability  of  the  extended  uncertain 
system  given  in  (30)  is  guaranteed  if  the  selected  Q  and 
R  satisfies  a  lower  and  an  upper  bound,  respectively. 

Theorem  3.5:  The  uncertain  system  in  (30)  is 
(M,  p)-stable  in  the  mean  if 

[or^n{Q)  +  CJ^n{R-')\m)N\t)\\  -  l}||P(/)ir^ 

>  2||AT(0II^  t  >  to,  (32) 
where  V{t)  satisfies  the  matrix  differential  equation 

V{t)  =  rit)Vit)  +  Vit)r\t)  +  |iei|/+  \\N(t)R-'Nitf\\I 
-  V(t)Ar^(t)Ar(t)V(t).  (33) 

Proof:  For  the  linear  time-varying  system  given 
in  (30),  uniform  asymptotic  stability  in  the  mean 
implies  (M,  y0)-stability  in  the  mean.  In  order  to  show 
the  uniform  asymptotic  stabihty  of  the  mean,  consider 
the  mean  dynamics  of  the  system  in  (30): 

fi^it)  =  t(0/«^(0  +  AT(0/«^(0,  (34) 

where  E[2(t)]  =  Hz{t)-  Construct  the  following 
Lyapunov  candidate  function: 

V[fiz{t)\  =  lil{t)V-\t)fi^{t).  (35) 

Note  that  the  solution,  V{t),  of  (33)  is  required  to  be  a 
bounded  positive  definite  matrix  as  long  as  AT  is 
norm-bounded  (Abou-Kandil,  Freiling,  lonescu,  and 
Jank  2003).  Thus  P"^(/)  exists  and  F[ai^(/)]>0  for  all 
fi^(t)^0.  Since  V(t)V~\t)  =  /,  the  time  derivative  of 
V(t)V-\t)  is  0: 

^[no^^-^o]  =  V{t)V~\t)  +  V{t)V~\t)  =  0. 

Solving  the  above  equation  for  V  ^  (0  and  substituting 
(33)  gives 

V~\t)  = -V-\t)V{t)V-\t) 

=  -v~\t)r(t)  -  T^{t)v~\t) 

-  [WQW  +  \\Nm-'N{tf\\}V~\t)V~\t) 
+  AT^(OAT(0. 


Now  the  time  derivative  of  (35)  can  be  written  as 
-nlr^V~'^iz  -[WQW  +  WNR-'N'^\\]nl 

+  filV-'[r^^  + AT  fi^] 
=  iijAT^p- +  filV-' AT  fi^ 

-{WQW  +  WNR-'N^w}tilv~^fiz-^f^l^r^^rfi^ 
=  fil^Ar^v~^+v-^Ar 

-{WQW  +  WNR-'N^w}'P~^ +Ar^Ar]fi^. 

Asymptotic    stability    in    the    first    moment  is 

guaranteed  if 

|at^P-i  +p-iAT 

-  {Iieil  +  WNR~^N^W}V~^  +  AT^At}  <  0. 
Note  that 

AT^-:p-ij|^AT^-P"^j^  >  0  =^  AT^AT  +  P"^ 

>  AT^V~^  -\-V~^AT. 

Thus  the  above  condition  for  asymptotic  stability  is 
satisfied  as  soon  as 

{2AT^AT  +  p-^  -  {lieil  +  WNR-^N^WjV'^]  <  0 

or 

|2^>AT^AT^>  +  /-  {llgll  +  IIA^i^-^TV^Il}/}  <  0. 
Using  the  inequalities 

ll^ll'l|AT||2/>^AT^ATP,   or„,in(G)<|iei|  and 
crmin(R-')WNN'^W  <  IIA^i^-'A^^II 
yields 

2||P||'||AT||2  <  a,nUQ)^^mm(R-')WNN'^W  -  1- 

Hence  the  uniform  asymptotic  stability  in  the  first 
moment  is  guaranteed  if 

2||AT(0f  <  {crmin(Q)  +  cr^n(R-')WNm\m  -  1} 

X  wmw-\  t>to. 

□ 

Remark  4:  The  uncertain  system  in  (30)  is  (M,  0)- 
stable  in  the  mean  if  the  selected  Q  and  R  satisfy  the 
inequaHty  in  (32).  Thus  for  a  highly  uncertain  system. 
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if  the  nominal  control  action  on  the  true  plant  would 
result  in  an  unstable  system,  then  selecting  a  small  Q  or 
a  large  R  would  also  result  in  an  unstable  closed-loop 
system. 


3.2  Mean  square  stability 

In  this  section  the  controlled  system's  stabiHty  in  the 
second  moment  or  the  mean  square  stability  is 
considered.  It  is  shown  here  that  the  (M,  yS)-stabiHty 
in  the  mean  implies  mean  square  stabiHty.  More  details 
on  mean  square  stabiHty  can  be  found  in  Kushner 
(1967)  and  Soong  (1973). 

Definition  3.6:  A  stochastic  system  of  the  following 
form  Z{t)  =  T(t)Z(t)  +  r(t)g(t)  is  mean  square 
stable  if 

lim  E[Z\t)Z(t)]  <  M,  (36) 

where  M  is  a  constant  square  matrix  whose  elements 
are  finite. 

Note  that  E[Z\t)Z(t)]  =  Tr{V(t)},  i.e. 

^E[Z{i)  Z\t)]  =  V{i)  =  T{i)V{i)  +  V{i)T^{i) 
at 

+  r(OA(or^(0 

and  the  solution  to  the  above  equation  can  be 
written  as 

V{t)  =    f      0(/,  T)r(T)A(T)r^(T)cD^(/,  T)dT 
J — oo 

The  (M,  yS)-stable  in  the  mean  impHes  that  the  system 
matrix,  T{t)  =  T{t)  +  AT{t),  generates  an  exponen- 
tially stable  evolution  operator  and  therefore  V{t)  has  a 
bounded  solution  (Abou-Kandil  et  al.  2003). 
Therefore,  for  the  system  given  in  (30),  (M,  )6)-stabiHty 
in  the  mean  impHes  mean  square  stabiHty. 


3.3  Almost  sure  asymptotic  stability 

The  solution  to  the  stochastic  system  given  in  (30) 
cannot  be  based  on  the  ordinary  mean  square  calculus 
because  the  integral  involved  in  the  solution  depends 
on  Q(t),  which  is  of  unbounded  variation  (Soong  and 
Grigoriu  1993).  For  the  treatment  of  this  class  of 
problems,  the  stochastic  differential  equation  may  be 
rewritten  in  Ito  form  as 

dz(t)  =  [r(t)Z(t)  +  Ar(t)Z(t)]dt  +  r(t)A^/\t)dB(t) 

or  simply  as 

dZ(t)  =  r(t)Z(t)dt  +  r(t)A^/\t)dB(t),  (37) 


where  dB(t)  is  an  increment  of  Brownian  motion 
process  with  zero-mean,  Gaussian  distribution  and 

E[dB(t)  dB\t)]  =  Idt.  (38) 

The  solution  Z(t)  of  (37)  is  a  semimartingale  process 
that  is  also  a  Markov  process  (Grigoriu  2002).  Details 
on  the  almost  sure  (a.s.)  stabiHty  for  the  stochastic 
system  in  (37)  is  presented  in  this  section. 

Definition  3.7:  The  linear  stochastic  system  given 
in  (37)  is  asymptotically  stable  with  probability  1,  or 
almost  surely  asymptotically  stable,  if 

P  (Z(t)  ^  0  as  ^  ^  oo)  =  1.  (39) 

(M,  y6)-stability  in  the  mean  response  impHes  that 
T(t)  generates  an  asymptotically  stable  evolution  for 
the  Hnear  system  in  (37),  but  it  does  not  imply  almost 
sure  asymptotic  stabiHty  due  to  the  persistently  acting 
disturbance.  In  fact,  given  T(t)  generates  an  asympto- 
tically stable  evolution,  the  necessary  and  sufficent 
condition  for  almost  sure  asymptotic  stability  is 

lim  ||r(Oll'log(0  =  0.  (40) 

A  detailed  proof  of  this  argument  can  be  found  in 
Appleby  (2002).  Equation  (40)  constitutes  the  sufficent 
condition  for  the  almost  sure  asymptotic  stabiHty  of 
a  linear  stochastic  system  given  (M,  )6)-stabiHty  in 
the  mean. 


4.  Stabilising     Q     and     transient     bound  on 
uncertain  system 

The  Lyapunov  analysis  given  in  Theorem  3.5  indicates 
a  lower  bound  requirement  on  the  system  process  noise 
covariance,  2,  and  an  upper  bound  requirement  on 
system  measurement  noise  covariance,  R,  in  order  for 
the  controlled  system  to  be  (M,  )6)-stable  in  the  mean. 
Since  the  measurement  noise  covariance  can  be 
obtained  from  sensor  calibration,  the  process  noise 
matrix  Q  is  usually  treated  as  a  tuning  parameter.  This 
would  compel  one  to  select  an  extremely  large  Q  so 
that  the  stability  is  always  guaranteed.  Selecting  a  large 
Q  value  would  force  the  estimator  to  completely  rely 
upon  the  measurement  signal  and  therefore  the  noise 
associated  with  the  measurement  signal  is  directly 
transmitted  into  the  estimates.  This  could  result  in  a 
noisy  control  signal  which  could  lead  to  problems  such 
as  chattering.  This  section  shows  a  systematic 
approach  to  select  a  stabiHsing  Q  using  the  over- 
bounding  method  of  Petersen  and  Hollot  (Petersen  and 
Hollot  1986;  Douglas  and  Athans  1994). 
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Assume  that  the  structure  of  the  uncertainty  AT(t) 
is  given  as 


AT(0  =  ;^r,(OT,-, 


(41) 


where  is  assumed  to  be  a  rank-one  matrix  of  the 
form  Ti  =  t/ef .  In  the  above  description,  r,(0  is  the 
ith  component  of  the  vector  r  g  9ft^  and  is  upper 
bounded  by 

r>sup|r/(OI    V/ G  {1,2,...,/}.  ^2) 

t>to  ^  ^ 

Define  matrices  T  and  E  as 

/  / 
T=J2^itf    and    E=J2^ief.  (43) 


i=l 


i=l 


Lemma  4.1:  If  the  uncertain  matrix  AT(0  has  the 
structure  given  in  (41),  then  the  following  matrix 
inequality  is  valid  for  all  matrices  V{t)  of  appropriate 
dimensions: 

:p*^(OAt^(0  +  at(o:p*(0 

<  ?^T-\-V*^(t)EP*(t)    Wt  >  to,  (44) 
where  r,  T  and  E  are  from  (42)  and  (43). 
Proof:    Substituting  T,  =  t,ef  into  (41)  yields 

Ar(t)v*(t)  +  v*^(t)Ar^(t) 


=  J2{rimefV*(t)  +  r,(t)V*\t)ettf}. 

i=i 


Note  that 


[r,m  -  P*^(0e/][r/(0t/  -  P*^(Oe/]^>  0. 


Thus 


r^mtf  +  p*^(Oe,efP*(0 


>  r,(/)t,e/P*(0  +  rt(t)V'\t)eitf 
J^ir-mtf  +  V*\t)etefV*(t)} 

i=l 

I 

>  ^{r,(Ot/efP*(0  +  r,(Or ^(Oe/tf}. 
i=i 

Now  substituting  for  7"  and  E  yields 

p^T-\-v*^(t)EP*(t)  >  Ar(t)v*(t)  +  v*^(t)Ar\ty 


and 


A  computationally  feasible  procedure  for  the 
calculation  of  a  stabiHsing  Q  is  given  next. 

Theorem  4.2:  Assume  that  the  uncertain  matrix  AT(^) 
has  the  structure  given  in  (41)  and  the  process  noise 
covariance,  Q*,  is  selected  so  that  the  following  matrix 
differential  Riccati  equation  has  a  bounded  positive 
definite  matrix  solution,  P*(0: 

V'^it)  =  r(t)V%t)  +  P*(Ot^(0 

-  yV\t)V\t)  +  7^(g*,  R-^)  (45) 

and 

n{Q'',R-^)  >  yV''(t)V*(t)-\-r^T-\-V*^(t)EP\t)  ^t>to, 

(46) 

where  y  is  a  positive  constant  and  7l(Q* ,  R~^)  denotes  a 
positive  definite  matrix  function.  Then,  the  uncertain 
system  in  (30)  is  (M,  P)-stable  in  the  mean  and 

M]  <  sup  am^^{V*(t))/arrnn{V*(to)),  (47) 

t>to 

where  represents  the  transient  bound  of  the  uncertain 
system's  mean  response. 

Proof:  Since  T(/)  is  assumed  to  generate  an  exponen- 
tially stable  evolution  operator,  there  exists  a  bounded 
positive  definite  matrix,  ^*(0,  that  satisfies 
Equation  (45).  Note  that  (45)  can  be  written  as 

V\t)  =  [t(0  +  AT(0]^*(0  +  ^*(0[T(0  +  AT(0]^ 

-Kp*(o^*(o  +  '7^(e^^"^) 

-  AT(OP*(0  -  V\t)Ar^(t). 
The  solution  to  above  equation  is 

V\t)  =  Ht,  to)V\to)^'^(t,  to) 

t 

^{t,x)[n{Q\R-^)  -  yV\T)V\x) 
-  AT(t):P*(t)  -  T>*(t)AT^(t)|o^(^,  r)dr. 


where  ^{t,  to)  is  the  evolution  operator  generated  by 
T(/)  =  t(/)  +  AT(/).  Based  on  Lemma  4.1  and  the 
matrix  inequality  Equation  (46) 

n{Q*,R-^) -yV*(t)V*(t) 

-  Ar(t)v*(t)  -  v*(t)Ar\t)  >  0. 


Thus 


□ 


V*(t)>Ht,  to)V\to)^'^(t,  to) 

>  0r„,in(P*(^))O(/,  to)^^(t,  to). 
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Now  (47)  follows  from 

^max 

(^*(0)/^mm(^*M>l|Oaro)||l 

Therefore,  to)  generates  an  exponentially  stable 
evolution.  □ 

Assuming  that  the  system  uncertainties  can  be 
written  in  the  form  given  in  (41),  a  stabilising  process 
noise  co variance,  Q*,  can  be  calculated.  Note  that 
bounds  on  the  system  uncertainties  used  here  may  be 
highly  conservative  and  therefore  it  may  result  in  an 
extremely  large  value  of  Q.  As  mentioned  earlier, 
selecting  a  large  Q  results  in  a  noisy  control  signal  and 
it  could  lead  to  problems  such  as  chattering.  Also  note 
that  obtaining  the  upper  bound  r  is  rather  difficult 
since  the  system  uncertainties,  AF(t)  and  AD(t),  may 
depend  on  the  estimator  gain,  K{t).  Thus  increasing  the 
process  noise  covariance  would  also  increase  the  upper 
bound  on  the  uncertainty,  i.e.  r.  Finally,  the  reader 
should  realise  that  the  dependency  of  system  uncer- 
tainties on  the  estimator  gain  is  eliminated  if  the 
control  distribution  matrix  is  precisely  known,  i.e. 
A^  =  0.  For  more  details,  please  refer  to  (18). 


5.  Adaptive  scheme 

After  substituting  the  DAC  law,  u(0  =  SZ(tX  the  plant 
dynamics  in  (1)  can  be  written  as 

X(0  =  AX(t)  +  8^SZ(t)  +      W(0]  (48) 

and  the  estimator  dynamics  in  (14)  can  be  written  as 

Z(0  =  [F^  +  D^S]Z(t)  +  mY(t),  (49) 


where 


Y(0  =  [Y(0-Y(0]. 


Let        Xext(0  = 


[Xf(0  X^(t)  W^(0]  ,  now  based  on  Assumption  3.1, 
the  controlled  plant  in  (48)  can  be  written  as 

Xext(0  =  AxtXext(0  +  ^ext^Z(0  +  W(0,  (50) 

where 


AqxI  — 


^2  0(„_^)x 
A4.  Irxr 


and  Bext  = 


B 

Orxr 


The  following  assumptions  are  now  made. 

Assumption  5.1:  77?^  pair  (Aext^^ext)  controllable 
and  the  pair  {Aq^u  H)  is  observable. 

Assumption  5.2:  There  exist  an  r  x  m  matrix  Tl  such 
that  n^n  > /^xm,  i-e.  m<r.  If  m>r,  then  the  r-outputs 
considered  here  are  selected  such  that  the  corresponding 
(^xt5  H)  is  observable. 


Assumption  5.3:  There  exists  an  mxm  matrix  R>  0 
such  that  Vt  >  to,  we  have 


'\nt)Y\t)\ 


>  R 


Based  on  Assumptions  5.1,  5.2  and  5.3,  an  adaptive 
scheme  for  selecting  the  stabiHsing  process  noise 
covariance  matrix  can  be  developed  as  shown  next. 

Theorem  5.4:    Given  Assumptions  5.1,  5.2,  and  5.3,  the 

controlled  system  is  mean  square  stable,  E\X{t)\  e 
L2nLoo  cind  X(t)  is  asymptotically  stable  in  the  first 
moment,  i.e. 

lim^[X(0]  =  0 

t^oo 

if  the  process  noise  covariance  is  updated  online  using  the 
adaptive  law 

dQ{t)  =  [AgQit)  +  Q(t)A^  +  )/nY(OY^(On^|d/, 

(51) 

where  Aq  is  an  r  xr  negative  definite  matrix  such  that 
0  <  — 2Tr{^g}  <  1  and  y  is  the  adaptive  gain. 

Proof  of  this  theorem  is  based  on  the  following 
lemmas. 

Lemma  5.5:  Consider  the  following  linear  stochastic 
system 

Z(t)  =  AZ(t)  +  U(0. 

If  the  matrix  A  generates  an  exponentially  stable 
evolution  operator  0^(/  —  to)  and  \J{t)  e  L2,  i.e. 


|U(r)|^drj 


1/2 


<  00, 


where  \  -  \  represents  the  Euclidean  norm,  then  Z(f)  G 
L^^L^  and 

lim  ^[Z(0]  =  0. 

t^OQ 

Proof:    The  solution  Z{f)  can  be  written  as 


Z(0  =  ^A(t  -  to)Z(t, 


0)  +  f  O 

J  to 


A(t-  r)U(r)dr. 


Since  0^(r  —  ^o)  is  exponentially  stable 

\\^A(t-to)\\<koe-''^'-''^  <ko  yt>to, 

where  ||  ||  represents  any  induced  matrix  norm  and 
and  a  are  two  positive  constants.  Thus 

\Z(t)\  <  \\^A(t  -  to)\\  \Z(to)\  +  f  \\^A(t  -  r)||  |U(r)|dr 

J  to 

<  Xoe-<'-''^\Z(to)\  +  r  Ao^"'^'"'^|U(r)|dr 

J  to 
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J  to 

The  last  inequality  is  obtained  by  expressing  e~''^^~'^^  as 

^-(a-ao/2)(t-r)^-ao/2(t-r)^    ^\^qj-q         <  2<3    is    a  pOSitive 

constant.  Applying  the  Schwartz  inequaUty  yields 


|Z(OI  <  Ao^-"(^-^°^|Z(^)|  +  Ao(^^^-(^^-^«^(^-^Mr^ 
X  |^^^-^o(^-^)|U(r)|2dr^  ^  . 


1/2 


^)|U(r)|Mr 

Thus 


^(2a  -  ao) 


(^^^-^«(^-^>|U(r)|2dr^ 


1/2 


and 


E[\m\]  <  he 


-a(t-to) 


^[|Z(ro)|] 


(£• 


,  1/2" 


-ao(t-x) 


^{la-ao) 
Therefore 

lim  ^[Z(0]  =  0. 
Also  note  that  £'[|Z(/)|]  is  bounded  by 
£[|Z(OI]  <  Xoe-<'-"''>Emm 


|U(T)|2dT 


1/2" 


Since  U(0  e  L2 


(^^J  |U(r)|2dr^  ^  j  <  ^[/J  lUWI^drj 


1/2 

|U(r)|Mr|  <oo. 


Note  that  Z{t)  e  L2  since 


1 


<  -  I  |U(r)|Mr, 
ao 


Finally  note  that 

Z(t)  e  L2  ^  E[Z(t)]  e  L2. 


Lemma  5.6:  Consider  the  following  linear  stochastic 
system 

Z(t)  =  AZ{t)  +  U(0, 
Y(0  =  CZ{t). 

If  {A,  C)  is  observable,  Y{t)  e  L2  and  \]{t)  e  L2,  then 
Z{t)  G  L2  n  Loo  (^nd 

limL[Z(0]  =  0. 

Proof:  If  (v4,  C)  is  observable,  then  there  exist  a 
matrix  K  such  that  Ao  =  A  —  KC  is  exponentially 
stable.  Now  Z(0  can  be  written  as 

Z(0  =  AZ(0  +  U(0  +  AT(0. 

Thus  from  Lemma  5.5  one  could  conclude  that 
Z(t)  G  L2  n  Loo  and 


limL[Z(0]  =  0. 


□ 


The  stability  analysis  given  in  Section  3  reveals  that 
selecting  a  sufficiently  large  process  noise  covariance 
would  guarantee  asymptotic  stability  of  the  controlled 
system's  mean  response.  Thus  the  adaptive  law  given  in 
Theorem  5.4  increases  the  process  noise  covariance  to 
ensure  that  L[Y(^)]  g  L2.  Now  based  on  the  above 
lemmas  the  proof  of  Theorem  5.4  can  be  easily 
obtained  as  shown  next. 

Proof:  Let  J^J  denote  a  filtration  generated 
by  Y(t),  i.e. 

l[y(^)|j^7]  =  y(^),  ^<  ^. 

Now  consider  the  following  nonnegative  function: 
F(^,^,Z,Y,P,  0 

=  £L[Y(r)|^^]^L[Y(r)|4]dr 

+  L[z(0|^t]^A^L[z(0|4] 
+  Tr{e*-e(0}+Tr{p; 


□ 


where  >  Q(t)  Vt  >  /q  is  a  stabihsing  process  noise 
covariance  and  P^^^  is  selected  such  that 
i'max  >  >  to,  where  P*(t)  may  be  obtained  by 

solving  the  continuous-time  matrix  differential  Riccati 
equation: 

P*(t)  =  FmP%t)  +  P%t)Fl  -  P*(t)H^R-^HP*(t) 
^GQ^G"^,    P*(to)  =  Po- 
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Note  that  for  any  Q(t)  <  g*,  P(t)  <  P^^^,  where  P(t) 
satisfies 

P(t)  =  FmP{i)  +  P{i)Fl  -  P{i)H^R-'HP{i) 

+  GQ{t)G^,    P(to)  =  Po-  (52) 

More  details  on  this  can  be  found  in  the  comparison 
results  given  in  Abou-Kandil  et  al.  (2003,  Chapter  4). 
The  matrix  A'  is  a  positive  definite  matrix  of  appro- 
priate dimensions  and  it  is  selected  so  that  it  satisfies 
the  following  matrix  inequahty: 

where  A^>0.  It  is  important  to  note  that  the 
expectation  given  in  the  above  non-negative  function 
is  conditioned  on  the  filtration  at  the  lower  time  limit. 
For  example,  consider^  a  time  instant  s  such  that 
toSs<t,  now  V{s,  t,  Z,  Y,  P,  Q)  can  be  written  as 

V(s,  t,Z,Y,  P,  Q) 

=  £^[Y(r)  I  ^j]^£[Y(r)  I  ^J]dr 

+  £[Z(0|^J]^A'£[Z(0|^J] 
+  Tr{e*-e(0}+Tr{p;,,-P(0}. 
Now  dV(s,  t,  Z,  Y,  P,  Q)  can  be  calculated  as 
&V{s,  r,Z,Y,P,  0 

=  £[Y(0|^j]^£[Y(0|^j]d^ 
+  E  [dZ(0  I  J^j]  ^XE  [z(0 1  J^j]  +  ^  [z(0 1  J^j]  ^ 
X  XE  [^dZ(0  I  J^j]  -  Tr{de(0}  -  Tr{dP(0}. 
Note  that 

E  [dZ(0  I  J^J]  =  [Fm  +  DmS]E  [Z(0  I  T^&t 
+  E[i^(0Y(0|j^j]d^. 

Thus 
dn^,  ^,Z,Y,P,  Q) 

=  E[Y(0|^j]^^[Y(0|^j]d^ 

-  Tr{de(0}  -  Tr{di>(0}  +  ^[z(/)  |  ^j]^ 

X  [F^  +  D^SYXE  [Z(0  I  J^j]d/ 

+  E  \K{t)Y{t)  I  ^j]  ^XE  [Z(0  I  J^jjdr 

+  E  [Z(0  I  ^J]  +  DmS]E  [Z(0  I  J^j]d/ 

+  E  [Z(0  I  J^j]  ^XE  \K(i)Y{i)  I  J^jjd^. 


Note  that  for  any  two  vectors  a  and  b  of  same 
dimensions,  the  following  inequahty  holds: 

a^a  +  b^b  >  a^b  +  b^a 

i.e. 

E  [Z(0  I  j^j]  ^XXE  [Z(/)  I  J^j]  +  ^  \K(t)Y{t)  I  J^j]  ^ 
x^[^(/)Y(/)|^j] 

>  ^  [^(/)Y(0  I  ^J]  ^A'^  [Z(0  I  ^J] 
+  ^  [Z(0  I  ^J]  ^A'^  [^(0Y(0  I  ^J] . 
Therefore 

dV{s,  ^,Z,Y,  P,  0 

<  ^  [y(/)  I  T^'^E  [y(0  I  ^J]d^  -  Tr{de(0} 

-Tr{dP(0}  +  ^[z(0|^j]^ 
X  [X[F^  +  D^S]  +  [F^  +  +  XX] 

X  F  [Z(/)  I  J^j]d/  +  F  [^(/)Y(/)  I  J^j]^ 
x£'[i^(OY(/)|j^j]d^. 
Now  employing  the  Cauchy-Schwarz's  inequality  gives 

dK(^,/,Z,Y,P,0 

<  f[y^(OY(0  I  T]Y^  +  E\^^{t)K^{t)K{i)Y{t)  I  J^jjd/ 

-Tr{de(0}+F[z(0|^j]^ 

X  [X[Fm+DmS]  +  [Fm+DmSYX  +  XX] 

xF[z(0|^j]d?-Tr{dP(0}. 
Substituting  (51)  and  (52)  yields 

dF(^,/,Z,Y,P,0 

<  f[y^(OY(0  I  T^dt  +  E\Y^{t)K^{t)K{t)Y{t)  |  T^&t 

-  E  [Z(0  I  J^j]  ^ME  [Z(0  I  j^j] d/ 

-  Tv[AQQ(t)  +  2(0^e  +  KnY(OY^(on^|d/ 

-Tv{F^P(t)  +  P(t)F^ 

-  P(t)H^R-^HP{t)  +  GQ{t)G^]dt. 

Note  that 

-Tr[^ee(o  +  ami]  =  -2Tr{^ee(o} 

<  -2Tr{^e}Tr{e(0} 
<Tr{e(0}. 


The  first  inequality  is  valid  because  —Aq  is  positive 
definite  and  the  process  noise  covariance  Q{t)  is 
positive  semi-definite  (Yang  2000).  The  last  inequaHty 
holds  since  0<  — 2Tr{^g}  <  1.  Also  note  that  due  to 
the  nature  of  matrix  G,  we  have 

Tr{GQ(t)G^}  =  Tr{e(0}. 

Thus 

dF(^,/,Z,Y,P,0 
<Tr|^[Y(OY^(0|^^]}d/ 
+  Tr{E[7(:^(0i^(0Y(0Y^(0 1  J^j]  }d^ 

-  E  [Z(0 1  J^j]  ^ME  [Z(0 1  T^dt 

-)/Tr|Y(OY^(On^n}d/  +  Tr{P(Oi/^i?"^i/P(OH/ 
-2Tr{F^P(0}d/ 
<  ^[(1  +  ||^(0^^(Oll)Tr  jY(OY^(0 j  I ^J]d/ 
-E\z{t)  I  J^j] ^ME\z{t)  I  J^jjd? 

-  )/Tr  I  Y(OY^(0  }  d?  +  i:x{P{t)H^R- ^  HP{t)  ]dt 
-2Tv[F^P{t)]dt. 

The  second  inequality  holds  since 

|Y(Ol'  +  l^(OY(Ol'  <  (1  +  ||^(0^^(0ll)|Y(0l'. 

Therefore  E[dV{s,  t,  Z,  Y,  F,  Q)  \  J^J]  can  be  written  as 
E^dVis,  ?,Z,Y, P,  e)|  J^j] 

<^[(i  +  ll^(0^^(Oll)|Y(Ol'|^j]d^ 

-  E  [Z(0  I  J^j]  ^A^^  [Z(0  I  T^dt 
-yE\\Y{i)\'\T^dt 

+  ^  ]j:T{P{t)H'^R-^HP{t)  -  2FmP{t)]  I  J^jjd^. 
Combining  the  similar  terms  yields 
E^dVis,  ?,Z,Y, P,  e)|  J^j] 

<  ^[(1  +  117^(0^^(011  -  y)|Y(Ol'  |^j]d^ 

-  £'[Z(0  I  J=-'^^ ME\±{t)  I  J^jjd? 

+  E]j:v[P{t)H^R-^HP{t)  -  2F^P{t)}  I  J^j]d/. 
Let  y  =  yi-\-  y2,  where  yi  is  selected  such  that 

n  >  i  +  ||^(Oi^'^(OII.  (53) 


Thus 

E^iVis,  t,  Z,Y,P,  e)|j^j] 

<-K2Tr|^[Y(0Y^(0|^j][d/ 

-  E  [z(o  I  j^j]  ^A^^  [z(o  I  j^jyt 

+  ^  [Tr{P(0/^^P-^/^P(0  -  2F^P(t)}  I  J^jjd^. 
Now  based  on  Assumption  5.3,  we  have 
E^dV(s,  t,  Z,Y,  P,  0|  J^j] 

<  -E  [Z(0  I  J^J^ME  [Z(0  I  J^j]d/ 
+  E[Tr{P(t)H^R-^HP(t)  -  2F^P(t)} 
-y2Tv{R}\j^J]dL 
Finally  note  that  E[dV(s,  t,  Z,  Y,  P,  Q)  |  J^J]  <  0  if 

yi  >  Tv{P(t)H^R-^HP(t)  -  2F^P(t)}Tv{R}~^ .  (54) 
Assuming  that  Y(^)  is  precisely  known  yields 

^[y(o|4]=^[y(o]. 

Thus  selecting  yi  and  y2  according  to  Equations  (53) 
and  (54)  yields 

E[v(to,  t,  Z,  Y,  P,  0]  -  V(to,  to,  Z,  Y,  P,  0 
=  ^'^[dF(^,r,Z,Y,P,0]  <0. 
Therefore 

E[v(to,  t,  Z,  Y, P,  0]  <  z\to)Xz(to)  +  Tr{e*  -  Q(to)} 
+  Tr{P;,,-P(/o)}. 

Also  note  that 

E  [  V(s,  t,  Z,  Y,  P,  0  I  j^j]  -  Z,  Y,  P,  0 

=      E  [d^^(^,  T,  Z,  Y,  P,  0  I  J^j]  <  0. 

Thus 

e[v(s,  t,  Z,  Y,  P,  0  I  j^j]  <  V(to,  s,  Z,  Y,  P,  0. 

Now  the  properties  of  V(to,t,Z,Y,P,Q)  may  be 
summarised  as 

(i)  K(/o,/,Z,Y,P,0>O; 

(ii)  4n^,/,Z,Y,P,0]<oo; 

(iii)  4V(^,^,z,Y,p,0i j^j] <  n^,^,z,Y,p,0, 

s<t;    ^  ^ 

(iv)  V(to,  t,  Z,  Y,  P,  Q)  is  adapted  to  J^J . 


These  properties  imply  that  V(to,  t,  Z,  Y,  P,  Q)  is  a 
non-negative  ^J-supermartingale  (Kushner  1967; 
Liptser  and  Shiryayev  1989)  and  the  non-negative 
supermartingale  probabihty  inequahty  yields 
(Doob  1953) 

P  (  sup  V(to ,  t,  Z,  Y,  P,  0  >  A. ) 

V  t>to  ) 

^  z^(/o)A-z(/o)  +  Tr{Q*  -  Q(/o)}  +Tr{P;;,,,  -  P(/o)} 
X 

where  A>0  is  any  positive  constant.  Thus  selecting 
sufficiently  large  A  yields 

P(  sup  n^o,  U  Z,  Y,  P,  0  <  oo  )  =  1 . 

That  is,  V(to,t,Z,Y,P,Q)  is  almost  surely  bounded. 
Note  that  V(to,  t,  Z,  Y,  P,  Q)  is  defined  as 


K(/o,^,Z,Y,P,  0 

=      E  [Y(r)]     [Y(r)]  dx  +  E  [z(o]  ^ XE  [z(o] 

+  Tr{G*-G(0}+Tr{p;,,-P(0}. 
Therefore 

n/o,^,Z,Y,P,0€Loo  a.s. 
=^  ^  p(0]  G  L2,    2(0  G  Loo   and  P(0  g  Loo  a.s. 

Since  P{t)  is  a.s.  bounded,  the  estimator  gain,  ^(0  = 
P(t)H^R~^,  is  also  a.s.  bounded.  Thus  there  exists  a  k* 
such  that 

P(sup||i^(0ll  >k*]  =0. 
The  estimator  dynamics  is  given  as 

Z(0  =  [F^  +  D^S]Z{t)  +  7^(0Y(0. 

Since  [iVz  +  D^^S]  generates  an  exponentially  stable 
evolution  operator,  and  since  E\Y{t)\  g  L2,  based 
on  Lemma  5.5,  it  can  be  shown  that  E[7u{t)\  g 

L2  n  Loo,  and 


lim  L 


[m]  =  0. 


Since  E[Y{t)\  g  L2, 

L[Z(0]gL2: 


'L[Y(0]gL2. 


Now  given  the  observabihty  assumption,  based  on 
Lemma  5.6,  it  can  be  shown  that  L[X(0]  G 
L2nLoo,  and 

lim  L[X(0]  =  0. 


Finally  note  that  the  controlled  closed-loop  system  can 
be  written  as 


"XextCO" 

'  XextCO 

_K{t)H   [F„  +  DrnS-K{t)H}_ 

.  z(o 

GV{i) 

Note  that  the  closed-loop  state  matrix 


(55) 


FcL(t)  = 


'AqxI  ^ext^ 

K(t)H  {F^^D^S-K(t)H}_ 


is  bounded.  Also,  the  asymptotic  stabiHty  of 
^[XextCO]  and  E[Z(t)]  impHes  that  the  matrix,  Pcl(0, 
generates  an  asymptotically  stable  evolution  operator, 
^ci.(t,  to),  i.e. 

lim  ||Ocl(^,^)||  =0. 
Equation  (55)  can  be  written  in  ltd  form  as 

dXcL(0  =  LcL(OXcL(Od/  +  rcL(OdHcL(0,  (56) 
where 


rcL(0  ■■ 


£[dBcL(OdB?L(0]  =  [^  ^] 


and 
dt  =  QcLdt. 


Remark  5:  It  is  important  to  note  that  if  one  wishes 
to  express  (55)  in  Stratonovich  form,  the  results  given 
here  hold  since  we  are  considering  linear  stochastic 
differential  equations  with  state  free  diffusion  term 
and  the  solution  obtained  from  the  Stratonovich 
integral  equation  converges  a.s.  and  uniformly  to  that 
obtained  from  the  Ito  integral  equation.  For  more 
details  please  refer  to  the  Wong-Zakai  theorem 
(Grigoriu  2002). 

Now  using  Ito  formula  d(XcL(OXcL(0)  can  be 
written  as 

d(XcL(OXjL(0) 

=  XcL(Od(XcL(0)^+  d(XcL(0)X?L(0 

+  rcL(OecLrcL(Od? 

=  {XclCOXJlW^'JlW  +  ^'cl(0Xcl(0X5l(0 

+  rcL(OecLrcL(0}d; 

+  rcL(OdBcL(OxjL(0  +  XcL(OdBjL(OrjL(0- 
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Figure  2.  Adaptive  DAC  block  diagram. 


Thus 

Xcl(OXJl(0 

=  *Cl(;,  fo)XcL(;o)XjL('o)<I>CL(''  'o) 
+  £  [XcL(T)[rcL(T)dBcL(T)]^]^ 

+  rXcL(T)[rcL(t)dBcL(r)]'' 

+  /"  <I>CL(f,  t)rcL(t)ficLrjL(t)<I>CL(^  ^)dr. 

J  to 

Therefore 

/•[XclCOXJlCO] 

=  E^^cLit,  ?o)Xcl(?o)XJl(?o)1'cl(?.  ?o) 

+  £  <I>CL(f,  r)rcL(T)eCLrjL(r)<I>jL(^  ^)drj. 

Since  <1>cl(^  to)  is  an  asymptotically  stable  evolution 
operator  and  FclCO  is  bounded,  it  can  be  easily  shown 
that  the  closed-loop  system  is  mean  square  stable,  i.e. 

lim  £[Xcl(OX?l(0]  <  M, 

t^OO 

where  M  is  a  constant  square  matrix  whose  elements 
are  finite  (Soong  1973).  □ 

Even  though  the  initial  process  noise  covariance, 
Q{to),  may  not  be  the  stabiHsing  Q,  the  adaptive  law 
given  in  (51)  can  be  used  to  update  the  process  noise 
covariance  online  so  that  the  controlled  system  is 
asymptotically  stable.  A  schematic  representation  of 
the  proposed  adaptive  controller  is  given  in  Figure  2. 

6.  Simulation  results 

For  simulation  purposes,  consider  a  two  degree-of- 
freedom  helicopter  that  pivots  about  the  pitch  axis  by 


4>  Yaw  Axis 
I 


Figure  3.  Two  degree-of- freedom  helicopter. 

angle  6  and  about  the  yaw  axis  by  angle  ij/.  As  shown  in 
Figure  3,  pitch  is  defined  positive  when  the  nose  of  the 
hehcopter  goes  up  and  yaw  is  defined  positive  for  a 
counterclockwise  rotation.  Also  in  Figure  3,  there  is  a 
thrust  force  Fp  acting  on  the  pitch  axis  that  is  normal  to 
the  plane  of  the  front  propeller  and  a  thrust  force  Fy 
acting  on  the  yaw  axis  that  is  normal  to  the  rear 
propeller.  Therefore  a  pitch  torque  is  being  applied  at 
a  distance  from  the  pitch  axis  and  a  yaw  torque 
is  apphed  at  a  distance  ry  from  the  yaw  axis.  The 
gravitational  force,  Fg,  generates  a  torque  at 
the  hehcopter  centre  of  mass  that  pulls  down  on  the 
helicopter  nose.  As  shown  in  Figure  3,  the  centre  of 
mass  is  a  distance  of  /cm  from  the  pitch  axis  along  the 
helicopter  body  length. 

After  hnearising  about  ^(^o)  =  i/(to)  =  0(to)  = 
i/{to)  =  0,  the  helicopter  equations  of  motion  can  be 
written  as 

(Jcq,p  +  mheli/cm)^"(0 

=  KppV^^pit)  +  KpyV^^yit)  -  BpOit)  +  W,{t)  (57a) 

(/eq,j+mheli/cm)V^(0 

=  KyyV^^yii)  +  KypV^^p{t)  "  Byf{i)  +   W2{i) .  (57b) 
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Table  2.  Two  degree-of- freedom  helicopter  model  parameters. 


System 

Assumed 

True 

parameter 

Description 

values 

values 

Unit 

Bp 

Equivalent  viscous  damping  about  pitch  axis 

0.8000 

1 

N/V 

By 

Equivalent  viscous  damping  about  yaw  axis 

0.3180 

-0.3021 

N/V 

Total  moment  of  inertia  about  yaw  pivot 

0.0384 

0.0288 

Kgm^ 

Total  moment  of  inertia  about  pitch  pivot 

0.0432 

0.0496 

Kgm^ 

Kpp 

Trust  torque  constant  acting  on  pitch  axis  from  pitch  motor/propeller 

0.2041 

0.2552 

Nm/V 

Kpy 

Trust  torque  constant  acting  on  pitch  axis  from  yaw  motor/propeller 

0.0068 

0.0051 

Nm/V 

Trust  torque  constant  acting  on  yaw  axis  from  pitch  motor/propeller 

0.0219 

0.0252 

Nm/V 

Kyy 

Trust  torque  constant  acting  on  yaw  axis  from  yaw  motor/propeller 

0.0720 

0.0684 

Nm/V 

Wheli 

Total  mass  of  the  helicopter 

1.3872 

1.3872 

Kg 

Location  of  centre-of-mass 

0.1857 

0.1764 

m 

A  detailed  description  of  system  parameters  and 
assumed  values  is  given  in  Table  2.  The  system 
states  are  the  pitch  and  yaw  angles  and  their 
corresponding  rates,  i.e.  0{t),  \l/(t),  9{t)  and  The 
control  input  to  the  system  are  the  input  voltages  of 
the  pitch  and  yaw  motors,  V^^p  and  V^^y,  respectively. 
The  external  disturbances  are  denoted  as  Wi{t)  and 

W2{i).  Let  x,{t)  =  [e{t)  mV,  ^2{i)  =  [e{t)  mf, 

<t)  =  [Vm,p{t)  V^,y(t)f  and  yV(t)  =  [Wi(t)  W2(t)f. 
For  simulation  purposes,  the  external  disturbance 
W(/)  is  selected  to  be 

Wi(t)  =  2.43^(0  - \3xlf(t)  -  Wi(t)  +  2W2{t)  +  Vi(0, 
W2{t)  =  -0.34^(0  +  1 .92xlf(t)  +  Wi(t)  -  3  W2(t)  +  ViCO 

(58) 

and 

rv,(0" 


=  V(t)  ~  A/'(0, 1  X  10-2/2x25(t)). 


Now  the  state-space  representation  of  the  plant  can  be 
written  as 

Xi(0  =  X2(0, 

Xiit)  =  ^4X2(0  +  Bu(t)  +  W(0,  (59) 

w(0  =  ^„,X2(0  +    w(0  +  v(/). 


where 

^W2 


2.43  -1.3 
-0.34  1.92 
ai  0 

0  (32 


] 


bi  b2 
h  b4 


and  the  system  parameters  are  given  as 


ax  = 


-By 


(/ec 


-  ^heli/cm)  ' 


b2 


(/eq,  J  +  ^heli/cm)  ' 
(jQq,p  +  ^heli/cm)  ' 


b3  = 


(Jcq,y  +  '^heli/cm)  ' 


b4  = 


(^eq,j+'^heli/cm)* 


The  state-space  representation  of  the  assumed  system 
model  is 

XiJt)  =  X2jtl 

X2.«  =  ^4^X2^(0 +  ^^U(/), 


where 


A.  = 


0 


0 

^2„,  J 


-[ 


The  measured  output  equations  are  given  as 

Y(0  =  CX(0  +  V(0, 

where  X(0  =  [Xf(0  Xf(/)]^  and  C  =  [/2x2  O2X2].  Note 
that  the  disturbance  term,  J>(t)  =  [V^(t)  V^(t)f, 
can  be  written  as 

V^(t)  =  AaiOit)  +  Abiui(t)  +  AZ?2W2(0  +  ^i(0 
V^(t)  =  /\a2f{t)  +  AZ73Wi(/)  +  AZ>4W2(0  +  W2{t). 
The  assumed  disturbance  term  dynamics  is  modelled  as 
V,^{t)  =  -V^^{t)  +  W,{t\ 

Let  the  extended  assumed  state  vector  be 
Zm{t)  =  [X^(0  ^^^(0  ^irS^)Y-  assumed 
extended  state-space  equation  can  be  written  as 


Zm{t)  =  Fm7.m{t)  +  D^u(i)  +  GW(0, 


where 


Fm  = 


"02x2 

^2x2 

02x2" 

"02x2 

02x2 

hx2 

,      Dm  = 

Bm 

_02x2 

02x2 

_02x2 

"04x2  1 

_/2x2j 

and 
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The  estimator  dynamics  can  be  written  as 

Z(0  =  F,,Z(t)  +  ^mu(0  +  m[CX(t)  -  HZ(t)] 

+  K(t)\(tX  (60) 

where  H=[C  02x2]-  The  nominal  controller  is  a  linear 
quadratic  regulator  which  minimises  the  cost  function 

1 


/  ((X^(0-x^)^2x(X^(0-x^) 
Jo 

+  u^(07^uu(0)d^  , 


(61) 


where  xj  =  [6d  i^d  0  0],  0^  and  xj/d  are  some  desired 
final  values  of  0  and  xj/,  respectively,  and  Qx  and  71^  are 
two  symmetric  positive  definite  matrices.  The  nominal 
control  that  minimises  the  above  cost  function  is 

where  is  the  feedback  gain.  Now  the  DAC  law  can 
be  written  in  terms  of  the  estimated  states  and  the 
estimated  disturbance  term  as 


=  SZ(t)  +  K^Xd. 


X(t)-Xd 


After  substituting  the  above  control  law,  the  true 
extended  system  dynamics  can  be  written  as 

02x2        ^2x2  02x2 
02x2        ^4^  ^2x2 
AvXt)  ^v^it)  Av,{t)_ 


"Xi(o" 

X2(0 

-Xi(o" 

X2(0 

_T>(t)_ 

02x6 

02x4 

"  02x1  " 

+ 

BmS 

m+ 

02x1 

Bx>2 

_W.(0_ 

(62) 


where 


Av,  (0  =  ABSK(t),   Av,(t)  =  A^2^4^  +  A^2, 

^1)3(0  =  A^2+^W3 


Bv,  (0  =  {ABS[F^  +D^S-  K(t)H] 
-\- AA2B^S  -  A^^ABS} 


and 


A^^ABKm 


Here      =  B  —  B^^  and  A^2  =  A4  —  A^^ . 

Table  3  shows  the  nominal  controller  and  estima- 
tor matrices.  Since  the  measurement  noise  co variance, 
R,  can  be  obtained  from  sensor  cahbration,  the  process 
noise  matrix,  g,  is  treated  as  a  tuning  parameter.  Based 
on  the  weighting  matrices  given  in  Table  3,  the 
feedback  gain  is  calculated  to  be 


r  ^• 


0229  0.8239 
0.8239  7.0229 


1.6691  0.3310 
-0.0830  2.4486 


For  simulation  purposes  the  initial  states  are  selected 
to  be  [6*0  to  Oo  xjfof  =  [-45°  0  0  0]^  and  the 
desired  states  0^  and  xl/^  are  selected  to  be  45°  and  30°, 
respectively. 

The  desired  response  given  in  Figure  4  is  the  system 
response  to  the  nominal  control  when  there  is  no 
model  error  and  external  disturbance.  For  illustrative 
purposes,  simulations  are  conducted  using  the 
traditional  DAC  as  well  as  the  proposed  adaptive 
DAC.  Results  obtained  using  the  traditional  DAC  is 
given  first. 


6.1  DAC  results 

Figure  5(a)  shows  the  unstable  system  response 
obtained  for  the  first  simulation  where  the  disturbance 
term  process  noise  covariance  is  selected  to  be  g  = 
10^  X  /2x2-  Figure  5(b)  shows  the  input  corresponding 
to  the  first  simulation  scenario.  Figure  6(a)  and  (b) 
contains  the  estimated  disturbance  term  and  the  error 
between  the  desired  states  and  the  true  states  corre- 
sponding to  the  first  simulation.  Note  that  the  first 
simulation  results  given  in  Figures  5  and  6  are  unstable 
due  to  the  low  value  of  Q  selected. 

A  second  simulation  is  conducted  using 
g  =  10^x/2x2-  The  system  response  obtained  for  the 


Table  3.  Nominal  controller/estimator  matrices. 


LQR  weighting  matrices 


Covariance  matrices 


7^u=10x/2x2 


2x=  1^ 


500  X  /2x2  02x2 
02x2  100  X  /2x2 


q3  q4 

10-^  X  /2x2  02x2 

02x2  ^2x2 

02x2  02x2 


7?=10-^x/2x 


02x2 
02x2 
10^  X  /2x2 
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Figure  4.  Desired  system  response. 


second  simulation  is  given  in  Figure  7.  Figure  8  shows 
the  estimated  disturbance  term  and  state  error 
obtained  for  the  second  simulation.  Note  that  the 
estimated  system  rates,  estimated  disturbance  term  and 
the  control  input  are  highly  noisy  because  of  the  large 
Q  selected. 

The  results  shown  here  indicate  that  for  a  small 
value  of  process  noise  covariance,  the  controlled 
system  is  unstable.  Though  a  large  value  of  process 
noise  covariance  stabihses  the  controlled  system,  it  also 
results  in  highly  noisy  estimates.  The  direct  dependency 
of  the  controlled  system's  stability  on  the  process  noise 
covariance  is  more  evident  in  the  simulation  results 
given  next. 

Combining  the  plant  dynamics  in  (59)  and  the 
estimator  dynamics  in  (60),  the  closed-loop  system 
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Figure  5.  (a)  Actual  states  and  (b)  input:  2  =  10^  x  /2x2- 
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Figure  6.  (a)  Disturbance  term  and  (b)  state  error:  Q=10  x  I2 


dynamics  can  be  written  as 


"Xi(o" 

'02x2 

^2x2 

02x2  ©2x6 

02x2 

/2x2  BS 

w(0 

02x2 

Ay/ 2 

^W3  ©2x6 

.^(0 

02x2 

02x2   (F„,  +  D„,S-K(t)H) 

-X|(o- 

02x1 

X2(0 

+ 

BKmXd 

w(0 

V(0 

-  Z(0  . 

(63) 


Since  we  are  considering  a  time-invariant 
system  here,  the  Kalman  gain  K{t)  converges  to  its 
steady-state  value  fairly  quickly.  Using  the  steady-state 
Kalman    gain,    the    stabihty    of   the  closed-loop 


system  can  be  easily  verified.  Figure  9  shows  the 
closed-loop  poles  of  the  system  for  different  values  of 
Q  ranging  from  1  x  10^  x  /2x2  to  1  x  10^  x  /2x2- 
Figure  9  indicates  that  the  controlled  system  is 
unstable  for  the  initial  small  values  of  Q  and  the 
closed-loop  poles  migrate  into  the  stable  region  as  Q 
increases. 


6.2  Adaptive  DAC  results 

Results  obtained  by  implementing  the  proposed 
adaptive  disturbance  accommodating  scheme  is  pre- 
sented in  this  section.  Based  on  the  assumed  system 
parameters  and  controller  design  matrices  given  in 
Tables  2  and  3,  the  assumed  state  matrix.  Am,  the 
assumed  input  matrix,  Bm  and  the  DAC  matrix, 


S,  can  be  calculated  as 

"0  0    1.00      0  ■ 
0  0      0  1.00 
0  0  -9.28  0 
0  0      0  -3.50 


■  0  0  ■ 

0  0 

2.37  0.08 

0.24  0.79 


and 


-7.02    -0.82    -1.67    -0.33  -0.42 
0.82     -7.02     0.08     -2.45     0.13  - 

As  shown  in  Section  6,  matrix  Ax>,^^  is  given  as 
-1  0 


0.04  "I 
-1.28J' 


Now  the  matrix  [F^ 


0  -3_ 
D^S]  can  be  calculated  as 


0 

0 

1.00 

0 

0 

0  - 

0 

0 

0 

1.00 

0 

0 

-16.56 

-2.50 

-13.22 

-0.98 

0 

0 

-1.04 

-5.756 

-0.34 

-5.51 

-0 

0 

0 

0 

0 

0 

-1.00 

0 

0 

0 

0 

0 

0 

-3.00 

Let  M  =  10  ^  X  /g,  now  the  positive  definite  symmetric 
matrix  X  that  satisfies  the  following  matrix  inequality: 

can  be  calculated  as 
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Figure  9.  System  closed-loop  poles  for  Q  varying  from 


Figure  10(a)  and  (b)  shows  the  system  response  and 
the  DAC  input  obtained  for  the  first  simulation. 
Figure  1 1(a)  and  (b)  contains  the  estimated  disturbance 
term  and  the  error  between  the  desired  states  and  the 
true  states  corresponding  to  the  first  simulation.  Note 
that  the  first  simulation  results  given  in  Figures  10  and 
1 1  indicate  that  the  adaptive  scheme  is  able  to  stabihse 
and  recover  the  desired  performance  despite  the  initial 
unstable  process  noise  covariance  selected.  The  time 


X  - 
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0.0513 

-0.0000 

_  -0.0000 

-0.0000 

-0.0000 

-0.0000 

-0.0000 

0.0167 

Since  the  number  of  inputs  and  the  number  of 
outputs  are  the  same  here,  the  matrix  n  is  selected  as 
the  identity  matrix  h^i-  For  the  implementation  of  the 
adaptive  law,  the  following  parameters  are  selected: 

Aq  =  —0.25  X  /2x2  and 

y  =  {\\K(t)\\  +  Tv{P(t)H^R-'HP(t) 

-2F^P(t)}Tv{R}-'^lO'). 

Three  different  simulation  scenarios  are  considered 
here. 

6.2.1  Simulation  I 

For  the  first  simulation,  the  initial  process  noise 
covariance    is    selected    to    be    2(^)  =  10~^  x /2x2- 


varying  process  noise  covariance  obtained  for  the  first 
simulation  is  given  in  Figure  12. 


6.2.2  Simulation  II 

For  the  second  simulation,  the  initial  process  noise 
covariance  is  selected  to  be  2(^)  =  /2x2-  Figure  13(a) 
and  (b)  shows  the  system  response  and  the  DAC  input 
obtained  for  the  second  simulation.  Figure  14(a)  and 
(b)  contains  the  estimated  disturbance  term  and  the 
error  between  the  desired  states  and  the  true  states 
corresponding  to  the  second  simulation.  The  simula- 
tion results  given  in  Figures  13  and  14  indicate  that  the 
adaptive  scheme  is  able  to  stabihse  and  recover  the 
desired  performance  despite  the  initial  unstable  process 
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Figure  10.  (a)  Actual  states  and  (b)  input:  Q(to)  =  10  hxi- 
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Figure  12.  Adaptive  process  noise  covariance:  Q{t^- 
10-^x72x2- 


noise  covariance  selected.  The  time  varying  process 
noise  covariance  obtained  for  the  second  simulation  is 
given  in  Figure  15. 

6.2.3  Simulation  III 

For  the  third  simulation,  the  initial  process  noise 
covariance  is  selected  to  be  =  10^  x  72x2-  Figure 
16(a)  and  (b)  shows  the  system  response  and  the  DAC 
input  obtained  for  the  third  simulation.  Figure  17(a) 
and  (b)  contains  the  estimated  disturbance  term  and 
the  error  between  the  desired  states  and  the  true  states 
corresponding  to  the  third  simulation.  Figure  18  shows 
the  time  varying  process  noise  covariance  obtained  for 
the  third  simulation. 

Figure  19  shows  the  time  varying  process  noise 
covariance  obtained  for  the  three  simulations. 
Figure  19  indicates  that,  regardless  of  the  initial 
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Figure  15.  Adaptive  process  noise  covariance:  2(?o)  =  /2x2- 


matrix  selected,  the  process  noise  covariance  settles 
down  at  its  steady-state  value,  which  is  around 
4  ~  5  X  10^  for  the  present  scenario. 


7.  Conclusion 

This  article  presents  the  formulation  of  an  observer- 
based  stochastic  DAC  approach  for  linear  time- 
invariant  MIMO  systems  which  automatically  detects 
and  minimises  the  adverse  effects  of  both  model 
uncertainties  and  external  disturbances.  Assuming 
that  all  system  uncertainties  and  external  disturbances 
can  be  lumped  in  a  disturbance  term,  this  control 
approach  utihses  a  Kalman  estimator  in  the  feedback 
loop  for  simultaneously  estimating  the  system  states 
and  the  disturbance  term  from  measurements. 
The  estimated  states  are  then  used  to  develop  a 
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Figure   18.  Adaptive  process   noise  covariance:    Q(tQ)  = 

10^x/2x2- 


nominal  control  law  while  the  estimated  disturbance 
term  is  used  to  make  necessary  corrections  to  the 
nominal  control  input  to  minimise  the  effect  of  system 
uncertainties  and  the  external  disturbances.  The 
stochastic  stability  analysis  conducted  on  the  con- 
trolled system  reveals  a  lower  bound  requirement  on 
the  estimator  design  parameters,  such  as  the  process 
noise  covariance  matrix  and  the  measurement  noise 
covariance  matrix,  in  order  to  ensure  the  controlled 
system  stabihty.  Since  the  measurement  noise  covar- 
iance can  be  obtained  from  sensor  cahbration,  the 
process  noise  matrix  is  treated  as  a  tuning  parameter. 
Based  on  the  stochastic  Lyapunov  analysis,  an 
adaptive  law  is  developed  for  updating  the  selected 
process  noise  covariance  online  so  that  the  controlled 
system  is  stable.  The  adaptive  scheme  introduced  here 
guarantees  asymptotic  stabihty  in  the  mean  and  the 
mean   square   stabihty   of  the   controlled  system. 
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Figure  19.  Adaptive  process  noise  covariance  matrices:  (a)  Q(to)=  10~^  x  /2x2;  (b)  Q(to)  =  hx2',  and  (c)  Q(to)=  10^  x  /2x2- 


The  simulation  results  given  here  explicitly  reveal  the 
direct  dependency  of  the  proposed  control  scheme  on 
the  process  noise  covariance  matrix.  Since  the  nominal 
control  action  on  the  true  plant  is  unstable,  selecting 
a  very  low  process  noise  covariance  resulted  in  an 
unstable  system.  On  the  other  hand,  selecting  a  large 
value  stabihsed  the  system  but  resulted  in  a  highly 
noisy  control  input.  The  numerical  simulations  indi- 
cate that  the  adaptive  scheme  is  able  to  stabilise  and 
recover  the  desired  performance  despite  selecting  an 
initial  unstable  process  noise  covariance.  The  results 
also  indicate  that  regardless  of  the  initial  matrix 
selected,  the  process  noise  covariance  settles  down  to 
its  steady-state  value. 
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Robust  Kalman-Bucy  Filter 

Jemin  George 


Abstract— De\e\o^meni  of  a  robust  estimator  for  uncertain  stochastic 
systems  under  persistent  excitation  is  presented.  The  given  continuous-time 
stochastic  formulation  assumes  norm  bounded  parametric  uncertainties 
and  excitations.  When  there  are  no  system  uncertainties,  the  performance 
of  the  proposed  robust  estimator  is  similar  to  that  of  the  Kalman-Bucy  filter 
and  the  proposed  approach  asymptotically  recovers  the  desired  optimal 
performance  in  the  presence  of  uncertainties  and  or  persistent  excitation. 

Index  Terms— Hoo  filtering,  Kalman-Bucy  filter. 


1.  Introduction 

Even  though  the  Kalman-Bucy  filter  possesses  some  innate  robust 
characteristics,  the  estimator  performance  degradation  in  the  presence 
of  system  uncertainties  may  not  be  tolerable.  The  robust  estimation 
problem  addressed  here  involves  recovering  unmeasured  state  variables 
when  the  available  plant  model  and  the  noise  statistics  are  uncertain. 
There  exist  several  literature  on  the  design  of  robust  estimators  based 
on  Hoc  filtering  [l]-[6],  where  the  estimators  are  designed  to  minimize 
the  worst  case  Hoo  norm  of  the  transfer  function  from  the  noise  inputs 
to  the  estimation  error  output.  Since  filtering  is  a  worst-case  design 
method,  while  guaranteeing  the  worst-case  performance,  it  generally 
sacrifices  the  average  filter  performance.  While  the  Hoo  formulation 
involves  deregularization,  a  robust  estimator  design  based  on  the  regu- 
larized least-squares  approach  is  presented  in  [7].  Extension  of  the  reg- 
ularized least  squares  approach  to  time-delay  systems  and  time  varying 
system  are  presented  in  [8]  and  [9],  respectively. 

Robust  estimation  approach  known  as  the  guaranteed  cost  filtering  is 
presented  in  [10]-[12].  Here,  estimators  are  designed  to  guarantee  that 
the  steady-state  variance  of  the  state  estimation  error  is  upper  bounded 
by  a  certain  constant  value  for  all  admissible  model  uncertainties.  De- 
sign of  robust  estimators  that  ensure  minimum  filtering  error  variance 
bounds  for  systems  with  parametric  uncertainty  residing  in  a  polytope 
are  given  in  [13]  and  [14].  A  robust  state  estimator  for  a  class  of  un- 
certain systems  where  the  noise  and  uncertainty  are  modeled  determin- 
istically  via  an  integral  quadratic  constraint  is  presented  in  [15].  This 
approach,  known  as  the  set-valued  state  estimation,  involves  finding 
the  set  of  all  states  consistent  with  given  output  measurements  for  a 
system  with  norm  bounded  noise  input  [16].  Petersen  and  Savkin  [17] 
provides  a  comprehensive  research  monograph  on  robust  filtering  for 
both  discrete  and  continuous  time  systems  from  a  deterministic  as  well 
as  Hex,  point  of  view.  More  recently,  [18]  and  [19]  proposes  a  robust 
filtering  approach  based  on  penalizing  the  sensitivity  of  estimation  er- 
rors to  parameter  variations. 

Presented  here  is  the  formulation  of  a  robust  estimator  for  uncertain 
linear  stochastic  systems  under  persistent  excitation.  The  proposed  es- 
timator guarantees  asymptotic  convergence  of  the  estimation  error  even 
in  the  presence  of  persistent  excitation.  In  contrast  to  the  existing  ap- 
proaches, the  main  contributions  of  this  manuscript  are  given  below: 

Manuscript  received  May  24,  2011 ;  revised  October  26, 2011 ;  accepted  May 
04,  2012.  Date  of  publication  June  06,  2012;  date  of  current  version  December 
17,  2012.  Recommended  by  Associate  Editor  M.  Lops. 
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Color  versions  of  one  or  more  of  the  figures  in  this  technical  note  are  available 
online  at  http://ieeexplore.ieee.org. 
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•  Most  of  the  existing  robust  estimator  schemes  are  in  discrete-time 
or  assumes  deterministic  noise  scenario.  Therefore,  a  complete 
stochastic  formulation  of  a  continuous  time  robust  estimator  is 
presented  here. 

•  Present  formulation  considers  uncertainties  in  both  system  and 
output  matrices  as  well  as  an  unknown  persistently  exciting  signal. 

•  Proposed  approach  asymptotically  recovers  the  desired  optimal 
performance. 

The  structure  of  this  technical  note  is  as  follows.  A  detailed  problem 
formulation  and  the  development  of  the  robust  estimator  are  first  given 
in  Sections  II  and  III,  respectively.  Afterwards,  numerical  simulations 
are  presented  in  Section  IV  to  further  illustrate  the  performance  of 
the  proposed  robust  estimator.  Finally  concluding  remarks  are  given 
in  Section  V. 

II.  Problem  Formulation 

Consider  the  following  linear  time-invariant  stochastic  process  de- 
fined on  the  filtered  probability  space  (Q,  J^,  {J^f },  P): 

dXt  =  {AXt+Wt}dt  +  dBt,       tG[fo,  cxd)  (1) 

with  initial  condition  Xf  q  =  Xo .  Here,  Xf  :  [f  o ,  cxd)  x  17  i— s-  5ft"  is  the 
state  vector,  Wf  :  [fo,  cxd)  x  17  i—s-  5ft"  is  an  unknown  persistent  exci- 
tation, and  Bt  :  [fo,  oc)  x  17  i— s-  5ft"  is  an  n-dimensional  -Wiener 
process  with  zero  mean  and  the  correlation  of  increments 

E  [{B(t)  -  B(0}  {B(t)  -  B(Of]  =  Q\t  -  C|. 

Elements  of  the  system  matrix  A  G  5ft" and  the  noise  intensity 
matrix  Q  G  5ft" ^",  are  assumed  to  be  unknown.  The  measurement 
process  is  given  as 

dYt  =  CXtdt  +  dVt  (2) 

where  Yf  :  [fo,  oc)  x  O  i— s-  5ft'^  is  the  output  vector,  and  Vt 
[fo,  oc)  X  17  I— s-  5ft'^,  is  assumed  to  be  an     -Wiener  process  with 
zero  mean  and  the  correlation  of  increments 

E  [{V(r)  -  V(C)}  {V(r)  -  V(Of]  =  R\t  -  C\. 

The  output  matrix  C  G  5ft'^^"  is  assumed  to  be  unknown  and  the 
measurement  covariance  R  G  5ft"^  ^    is  assumed  to  be  known. 

Definition  1:  The  stochastic  process  Wf  is  persistently  exciting 
with  level  a  [20]  if 

P  (^j^'^^  W.W^rfT  >  aT/„xn  j  =  1  (3) 

for  some  a  >  0,  T  >  0,  and  Vf  >  f o . 

Since  the  true  system  state  matrix.  A,  and  the  output  matrix,  C,  are 
unknown,  the  assumed  (known)  system  matrix  and  output  matrix  are 
given  as  Am  and  Cm,  respectively.  Define 

Mt  =  {A-Am}Xt-\-  Wi,  and  (4) 
Nt  =  {C-Cm}Xt.  (5) 

Now  the  system  equation  and  the  output  equation  may  be  rewritten 
in  terms  of  the  assumed  system  matrix,  Am ,  and  the  assumed  output 
matrix.  Cm,  as 

dXt  =  {AmXt  +  Mt}  dt  +  dSt,  (6) 
dYt  =  {CmXt  +  Nt}  dt  +  dVt.  (7) 


Assumption  1:  The  persistent  disturbance,  Wf,  is  an  -adapted 
process  and  is  almost  surely  (a.s.)  upper  bounded  as  follows 

P  (sup  |Wt|  <  cx:  )  =  1  (8) 

\t>io  J 

where  |  ■  |  is  the  one-norm,  i.e.,  |x|  =  ^  -  \xi\ 

Assumption  2:  The  matrices,  A  and  Am  are  Hurwitz.  The  assump- 
tion of  the  stability  of  the  uncertain  system  is  a  weak  one,  as  it  is  rare 
that  an  estimator  be  applied  'open-loop'  to  an  unstable  system  [10]. 
Furthermore,  if  A  is  only  marginally  stable,  and  the  the  persistent  ex- 
citation is  such  that  the  system  states  are  bounded,  then  an  equivalent 
representation  of  system  (1)  similar  to  the  one  in(6)  can  be  selected 
such  that  the  state  matrix  is  Hurwitz. 

Lemma  1:  Given  assumptions  1  and  2,  the  process,  Xf ,  is  a.s.  upper 
bounded  as  follows: 


where  c  <  cx:  is  a  constant. 

Proof:  Matrix  A  generates  an  exponentially  stable  evolution  op- 
erator $A(f  —  fo)  and  the  solution  of  (1),  X(f),  can  be  written  as  [21] 

X(f)  =  ^A{t  -  fo)X(fo)  +  f  ^A{t  -  r)W(r)dr 

+  /  ^A{t-T)dB{T). 

Jto 

Since  ^^(f  —  fo)  is  exponentially  stable 

II  ^A(t  -  fo)  ||<  Aoe-'^^*-*^^  <  Ao,    Vf  >  fo 

where  ||  ■  ||  represents  induced  matrix  norm  and  Ao  and  a  are  two 
positive  constants.  Now  define 

Mt  =   f  ^A(t-T)dB(T). 

Note  that  M.t  is  a  supermartingale  and  based  on  Doob's  martingale 
inequality  [22]  it  is  a.s.  bounded.  Thus 

|x(f)|<  \\^A{t-u)\\\Mt.)\ 

+   f  \\^A(t-T)\\\W(T)\dT+\Mtl 

<Aoe-"^*-*o>|X(fo)| 

+  rAoe-"^*-^>|W(T)|rfT+|jW,|, 

Jto 

<Aoe-"^*-*o>|X(fo)| 

+  /  Aoe-^'^-'^o^'>^*-">e-'^o^'^*-">|W(T)|rfT 

Jto 

+  \J^t\. 

The  last  inequality  is  obtained  by  expressing  e~"^*~^^  as 
^-(a-ao/2)(t-T)^-ao/2(t-T)^^^^^^      ^  2a.  is  a  posltlvc  coustaut. 

Applying  the  Schwartz  inequality  yields 
|X(f)|  <Aoe-"^*-*°>|X(fo)| 

at  N  1/2 

^-(2a-ao)(t-r)^  \ 

/    pt  n1/2 

-\-\j^t\. 
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Thus 


|X(f)|  <  Aoe-"^*-*o^|X(f, 


Now  assume  for  a  minute  that  there  are  no  uncertainties,  i.e.,  Am  = 
A,  Cm  —  C,  Wf  —  0,  and  Bt  —  Bt,  where  Bt  is  an  J?^f -Wiener 
process  with  zero  mean  and  known  correlation  of  increments  Q 


+ 


Ao 


j'^  e--oii-r)^w(TtdT^ +  \M,v        E  [{B{r)  - m]  {^w  - mf]  =  Q\r  -  ci. 


Given  |Xo  |  is  a.s.  bonded,  based  on  assumption  1,  it  can  be  shown  that 
there  exist  a  positive  constant  c  <  oc  such  that 


P    sup  |Xi|  <c)  =1. 

\t>to 


Given  the  system  parameter  uncertainties  are  bounded  and  the 
system  states  are  a.s.  bounded,  an  upper  bound  on  Mt  can  be  obtained 
as 


P(|Afi|  </7(i))  =  l,    Vf  >fo. 


(9) 


Consider  the  process  Nt  defined  in  (5),  now  the  trivial  application  of 
Ito  formula  [23]  on  Nt  yields 

dNt  =  {C  -  Cm}{Amyit  +  Mt}  dt  +  {C-  Cm}  dBt.  (10) 

Define 

Ut  =  {C  -  Cm}  {[Am  +  /nxn]  X^  +  Mt}  .  (11) 

Since  the  system  parametric  uncertainties,  system  states,  and  Mt  are 
assumed  to  be  upper  bounded,  a  conservative  upper  bound  on  the  sto- 
chastic process  U t  can  easily  be  obtained,  i.e. 


(12) 


After  appending  dNt  to  the  system  equation  the  extended  system  may 
be  written  as 


Am  0 

0  -/ 


Mt 
Ut 


dt- 


I 

AC 


dBt  (13) 


where  AC   =   {C  -  Define  Zi   =   [Xf  Nff,F 


Am  0 
0         —ImXm  ^ 

may  be  written  as 


G-- 


AC 


and  Tt  =  [  Af  f    Uf  f.  Now  (13) 


dZt  =  {FZt-\-rt}dt-\-GdBt 
and  the  output  equation  may  be  written  as 

dYt  =  HZtdt  +  dVt 


(14) 


(15) 


where  H  =  [  Cm  i^m  x m  ] •  Note  that  the  process  Ft  =  [  AlJ  Uj  ]^ 
may  be  upper  bounded  by  j(t),  where 


j{t)  =  max  {//(t),    '^(t)}  • 


(16) 


Remark  1:  In  practice,  an  upper  bound  on  the  system  states  and 
the  disturbance  can  be  obtained  from  knowing  the  regime  and  the  en- 
vironment in  which  the  system  is  operating.  An  upper  bound  on  pa- 
rameter uncertainties  can  be  obtained  from  system  identification/cali- 
bration [24],  [25].  After  substituting  these  bounds  into  (4)  and(ll),  an 
upper  bound  on  Tt  can  be  obtained  from  (16). 


Then,  Tt  is  identically  zero  for  all  time  and  the  system  in  (14)  can  be 
written  as 


dZmt  =  FZmtdt  -\-  GmdBt 


(17) 


where  Cm  =  [Inxn  Onxm]^.  The  corresponding  measurement 
process  is  given  as 


dYm,  =  HZm,dt-\-dVt. 


(18) 


For  the  system  in  (17),  an  optimal  estimator,  such  as  a  Kalman-Bucy 
filter  [26],  of  the  following  form  can  be  designed: 

dZm,  =  FZmtdt  +  K  \dYmt  "  i?Z^,rftl  (19) 


where  K  G  is  the  steady-state  Kalman  gain  and  is  calcu- 

lated as 

K  =  PH^R-^  (20) 

where  P  G  can  be  obtained  by  solving  the  algebraic 

Riccati  equation  [27]: 

FP    PF^  -  PH^ R-^ HP  ^  GmQGm  =0.  (21) 

Now  subtracting  (19)  from(17),  the  estimator  error  dynamics  corre- 
sponding to  the  optimal  estimator  in(19)  may  be  written  as 

dim,  =[F-  KH]  Zm, dt  -  KdVt  +  GmdBt  (22) 

where  Zmt  —  Zmt  —  Zm-t  -  Note  that  the  estimation  error  is  unbiased, 
,  and  the  steady-state  value  of  the  error  covariance 


i.e.,  E  \ZmA  =  0,  J 
is  given  as  [26] 


(23) 


Given  next  is  the  formulation  of  a  robust  estimator  which  guarantees  the 
asymptotic  convergence  of  the  true  state  estimation  error  to  the  desired 
optimal  error  Zmt- 

III.  Robust  Estimator 
In  this  section,  a  robust  estimator  of  the  following  form  is  proposed: 

rfZi  =  \FZt  +  n{i)\di  +  K  \dYt  -  EZtd^  (24) 

where  J?(f)  is  a  signal  whose  details  would  be  explained  shortly.  The 
robust  estimator  formulation  presented  here  follows  the  typical  robust 
controller  formulation  [20]  in  that  the  estimator  "input,"  J?(f),  is  se- 
lected such  that  the  true  estimator  error  dynamics  given  in  (25)  asymp- 
totically tracks  the  desired  error  dynamics  given  in  (22).  A  systematic 
approach  for  the  selection  of  f){t)  constitutes  the  main  result  of  this 
section  and  is  presented  in  Theorem  1. 

Define  Zf  =  Zf  —  Zf ,  now  subtracting  (24)  from(14),  the  estimator 
error  dynamics,  i.e.,  rfZf,  may  be  written  as 

dit  =  {[i^  -  KE\  Zt^Vt-  -  KdVt  +  GdBt.  (25) 
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Define  Zt  as  the  difference  between  the  desired  estimation  error  in  where  sgn{-}  denotes  the  signum  function  or  the  sign  function  and  7(f) 
(22)  and  the  true  estimation  error  in  (25),  i.e.,  Zt  —  T^t  —  T^m-t  •  After     is  an  upper  bound  on  the  process  Ff . 

subtracting  (22)  from  (25),  dZt  can  be  written  as  Proof:  Proof  of  this  theorem  is  based  on  the  stochastic  Lyapunov 

stability  analysis.  Consider  a  stochastic  Lyapunov  function 


dZt  =  {[F  -  KH]  Zt+Tt-  n{t)}  dt  +  GdSt  -  GmdSi.  (26) 
The  solution  Zt  of  (26)  can  be  written  as 


V{Z,)  =  z^.xz,. 


Zt^Z^+  /   {[F  -  KH]  Zs+Ts-  ■n{s)}  ds 

Jto 


Let  Yf  denotes  the  innovations  process,  i.e. 


Now  d.V(Zt)  can  be  calculated  as 
d.V(Zt)  =  if  |[F  -  KHf  X  +  X[F-  KH]^Ztdt 
+  /  GdBs  -  f  GmdBs.{21)  +2E  \z^,  XTt^T?]  dt  -  2E  A'i?(t)|^? 

Substituting  (30)  and  (31)  yields 


dt. 


r  .     n  dV{Zt)  <  -zJsZtdt  +  2E  \zjH^A^TtyT]  dt 

dYt=  \dYt-HZtdt\.  V     /_      t  L  *  I  *  J 

^  ^  -2E[zjH^A^n(t)\^r]dt. 


Note 


Now  define  Zt  —  E  \Zt        '  where       is  the  filtration  generated 

by  Yf[28].  Note  E  \^dBt\Tt^  =  0  and  =  0,  thus  Zt 

can  be  written  as  HZtdt  —  E  ^^Zt\^Tf^  dt 

Z,  =  Z,  +  E\f  \[F-KH\Z,+T,-rt{s)]dsyr].  (28)  =E[{HZ,dt  -  H'L^Jt)\r7 


Based  on  previously  stated  assumptions,  we  could  presume  that  Zt  is 
mean  square  Riemann  integrable^  and  therefore,  Zt  can  be  written  as 


-dYt-E 


[dYra^lj^T] 


Zt^Zo-^  r  [F  -  KH]  Zsds  +  f  E  [f, ds  where  dY^,  =  [dYm,  -  HZraJt^  •  Given  Zr„(fo)  =  Zr„(fo),  one 

could  conclude  that  Ymt  is  a  zero  mean  processes  and 


*0 


.  [E[n(s)\:FT]ds.  E[dY^,\T7]=E[dY^,] 


Thus,  dZt  can  be  written  as  Therefore  dV{Zt)  can  be  bounded  as 

Zt  =  [F-  KH]  Ztdt  +  E  [Ttlj"?]  dt-E  [}?(f ) \Tf]  dt.   (29)     dV{Zt)  <  -zj SZtdt 


Theorem  1:  Assume  there  exist  a  positive  definite  symmetric  matrix 

X  G  ^(r^+m)x(n+m)  ^^j^h  that 

[F  -  KH]^  X  +  X[F-  KH]  +  5  <  0  (30) 

and 

X  =  H^A^  (31) 


+2E 


dYfA^Tt  -  dYfA^rtit)  1^?^ 


After  substituting  (32),  an  upper  bound  on  dV{Zt  )  can  be  obtained  as 

dV(Zt)  <  -zJsZtdt 


+2E 

Note 


dYjA^Tt  -  rfYf  A^sgn  [AdYt]  7W||^f^j  ■ 


where  5  G  3f?^"+"'^''^"+"'Ms  a  positive  definite  matrix  and  A  G 

<^{n+m)xm^  Thcu  thc  robust  estimator  error  dynamics  in  (25)  is  glob-  dYf  A^sgn  |ArfYt |  =  \AdYt\ 

ally  asymptotically  stable  in  the  first  moment,  i.e. 


lim  E  TZfl  -  0 


mvddYtA^Tt  <  \dYtA^\\Tt\  <  |rfY?^A^|7(t).  Thus 
dV(Zt)  <  -ZjsZtdt. 


and  we  have  mean  square  convergence  of  the  true  conditional  error  to 

the  desired  conditional  error,  i.e.  Now  it  can  be  concluded  that  for  every  initial  value  Z(to),  the 


lim  E 

t  — >  oo 


solution,  Zt,  of  (29)  has  the  property  that  Zt  ^0  almost  surely  as 
[iZtf]  =0  f^cxD[29],i.e. 


if  the  estimator  inputs,  ,(t),  are  selected  as  ^  C        '^^1  =  ^)  = 

n(t)  =  sgn  [AdYt }  j(t)  (32)     Also  note  that,  Vf  >  f □ ,  we  have 

iSee  [26,  Theorem  3.8]  V{Zt)  <  V{Zo)  ^  E  [v{Zt)]  <  E  [v{Zo)]  <  oo. 
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Fig.  1.  Persistently  exciting  disturbances,  (a)  Wi^ .  (b) 


The  design  of  proposed  robust  estimator  requires  the  calculation  of 
matrices  X  and  A  that  satisfies  the  conditions  given  in  (30)  and(31). 
As  shown  in  [31],  matrices  A'  and  A  that  satisfies  the  conditions  (30) 
and(31)  can  be  obtained  by  solving  the  following  linear  matrix  in- 
equality: 


{F  -  KHY  ^  - 


■  x{F  -kh}- 

-  KH 


0, 


X-H'  A' 

(n+m)  X  (n+m) 


<  0.  (33) 


Matrices  that  satisfy  the  above  LMI  can  be  obtained  by  using  the 
MATLAB  LMI  toolbox  [31],  [32]. 


-  - 
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Fig.  2.  Simulation  I:  State  estimation  error,  (a)  Kalman-Bucy  Filter,  (b)  Robust 
Kalman-Bucy  Filter. 
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Fig.  3.  Simulation  I:  State  Estimation  Error  Norm,  (a)  Kalman-Bucy  Filter, 
(b)  Robust  Kalman-Bucy  Filter. 


Thus  Vf  >  fo,  Zt  e  Cl\  i.e.,  E  \Zt 
conclude  that 


sup  \Zt\ 
t>to  '  ' 


<  oc.  Therefore,  we  could 


<  CXD. 


Now  based  on  the  Lebesgue's  dominated  convergence  theorem,  a.s. 
convergence  of  Zt  implies  [30] 


lim  E 

t  — >  oo 


Moreover,  Zt  G      implies  Zt  £  and 


lim  E\\Zt\\ 

t  — >  OO       L  J 


Note 


E[\Zt\\  >  \E[Zt]  I  =  l^l^i] 


Thus,  the  asymptotic  convergence  in  the  mean  follows  from  the  fact 
that  Zrnt  is  a  zero  mean  process.  ■ 


IV.  Numerical  Simulations 

Performance  of  the  proposed  estimator  is  evaluated  here  though 
numerical  simulations.  For  simulation  purposes  consider  a  stochastic 
system  of  following  form: 

-1.000  0 

0  -2.000 
-8.500  -1.620 
-1.020  -9.500 


0 
0 

2.300 
1.780 

+[0ix2 
+[0ix2 


0 
0 

1.330 
2.030 

Wu 


dt 


dt 


The  measurement  equation  is  of  the  following  form: 


IY4J 

0.4300 
0 

3.640 
0 


0 

-2.400 
2.740 
2.420 


-2.310 
0 
0 

1.120 


0 

1.540 
0 

4.340 


^1. 


dt 


The  assumed  matrices  are  selected  as 


0 

0 

-1.000 

0 

0 

0 

0 

-2.000 

0.300 

0.130 

-4.240 

0.200 

-0.400 

0.530 

0.420 

-5.430 

and 

0.430 

0 

-2.310 

0  - 

0 

-2.400 

0 

1.540 

0.950 

1.700 

0 

0 

0 

0 

1.300 

0.430 

The  processes  Wi^  and  IFa^  are  given  in  Fig.  1.  Note  that  the  system 
uncertainty  is  only  associated  with  the  dynamics  of  the  last  two  states 
and  the  output  matrix  error  is  only  associated  with  the  last  two  outputs. 
Since  the  two  measurement  uncertainties  are  only  associated  with  the 
last  two  outputs,  Nf  G      and  thus  the  matrices  F  and  H  are  defined 

as  F  = 


and     =  [  Cm 

-  [^2X2      02x4  ] 


[O2 


^2x2]^  ],  respec- 


,  the  steady-state  Kalman 


04X2 
_02x4  —I2X2_ 

tively.  After  defining  Gm 
gain  can  be  calculated. 

Since  there  is  no  uncertainties  or  external  disturbances  acting  on  the 
first  two  states  Tt  is  defined  as  Ff  =  E  [  Mi^  M^^  UT^  ^^?t\^  ^ 
where  H  =  [O4X2  /4x4]^.  Now  the  robust  estimator  may  be  de- 
signed based  on  the  premises  of  Theorem  1  after  replacing  the  con- 
dition X  =  with  X'E  =  .  System  process  noise  co- 
variance  and  the  measurement  noise  covariance  matrices  are  selected 


0  0  35  '  diiierent  sim- 
first  simulation  scenario  considers 


as  7?  =  10  ^  X  J4X4  and  Q  = 

ulations  are  considered  here.  The 
the  case  where  there  is  no  system  uncertainties.  For  the  second  sce- 
nario, aforementioned  uncertainties  as  assumed.  True  initial  states  are 
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Fig.      4.  Simulation 
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Fig.  5.  Simulation  II:  State  estimation  error,  (a)  Kalman-Bucy  Filter, 
(b)  Robust  Kalman-Bucy  Filter. 
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Fig.  6.  Simulation  II:  State  estimation  error  norm,  (a)  Kalman-Bucy  Filter, 
(b)  Robust  Kalman-Bucy  Filter  . 


selected  tobeXo  =  [l  —2  5  0]^  and  the  assumed  initial  states 
are  X^^  —  O4XI. 

A.  Simulation  I 

For  the  first  simulation,  consider  a  scenario  where  there  is  no  model 
uncertainties  and  external  disturbances,  i.e.,  A  =  Am,  C  =  Cm 
and  Wf  =  0.  For  the  first  simulation,  the  state  estimation  error  ob- 
tained using  the  Kalman-Bucy  filter  is  given  in  Fig.  2(a)  and  the  esti- 
mation error  obtained  from  the  proposed  robust  estimator  is  presented 


in  Fig.  2(b).  Fig.  3  shows  the  estimation  error  norm  obtained  for  both 
estimators.  Results  shown  in  Figs.  2  and  3  indicate  that  the  performance 
of  the  proposed  robust  estimator  is  identical  to  that  of  the  Kalman-Bucy 
filter  when  there  is  no  system  uncertainties. 

B.  Simulation  II 

The  uncertain  system  is  considered  for  the  second  simulation.  Fig.  4 
contains  the  model-error  vector,  Tt,  corresponding  to  the  aforemen- 
tioned uncertainties.  Note  that  at  any  time,  \Tt\  is  upper  bounded  by 
7  =  8.  The  state  estimation  error  obtained  using  the  Kalman-Bucy 
filter  is  given  in  Fig.  5(a)  and  the  estimation  error  obtained  from  the 
robust  estimator  is  presented  in  Fig.  5(b).  Fig.  6  shows  the  estimation 
error  norm  obtained  for  both  estimators.  Results  shown  in  Figs.  5  and  6 
indicate  that  the  performance  of  the  proposed  robust  estimator  is  supe- 
rior to  that  of  the  Kalman-Bucy  filter  when  there  is  system  uncertain- 
ties. Finally  note  that  the  performance  of  the  robust  estimator  given  in 
Figs.  5(b)  and  6(b)  are  similar  to  the  ones  given  in  Figs.  2(b)  and  3(b). 

V.  Conclusion 

This  technical  note  presents  the  formulation  of  a  robust  estimator 
for  uncertain  stochastic  systems  under  persistent  excitation.  The  con- 
tinuous-time stochastic  formulation  given  here  assumes  norm  bounded 
parametric  uncertainties  and  excitations.  Proposed  robust  estimator 
guarantees  the  asymptotic  convergence  of  the  state  estimation  error. 
Simulation  results  given  here  indicate  that  the  performance  of  the 
proposed  robust  estimator  is  similar  to  that  of  the  Kalman-Bucy  filter 
when  there  are  no  system  uncertainties  and  excitations.  As  shown 
by  the  numerical  simulation  results,  the  approach  asymptotically 
recovers  the  desired  optimal  performance  in  the  presence  of  system 
uncertainties  and  persistent  excitation.  Potential  future  work  include 
extending  the  proposed  approach  to  the  nonlinear  estimation  problem 
by  introducing  constrains  on  the  drift  and  diffusion  terms  of  the 
nonlinear  stochastic  differential  equation. 
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Abstract 

Many  applications  of  autonomy  are  significantly  complicated  by  the  need  for  wireless  networking,  with  challenges  includ- 
ing scalability  and  robustness.  Radio  accomplishes  this  in  a  complex  environment,  but  suffers  from  rapid  signal  strength 
variation  and  attenuation  typically  much  worse  than  free  space  loss.  In  this  paper,  we  propose  and  test  algorithms  to 
autonomously  discover  the  connectivity  area  for  a  base  station  in  an  unknown  environment  using  an  average  of  received 
signal  strength  (RSS)  values  and  a  RSS  threshold  to  delineate  the  goodness  of  the  channel.  We  combine  region  decom- 
position and  RSS  sampling  to  cast  the  problem  as  an  efficient  graph  search.  The  nominal  RSS  in  a  sampling  region  is 
obtained  by  averaging  local  RSS  samples  to  reduce  the  small-scale  fading  variation.  The  RSS  gradient  is  exploited  dur- 
ing exploration  to  develop  an  efficient  approach  for  discovery  of  the  base  station  connectivity  boundary  in  an  unknown 
environment.  Indoor  and  outdoor  experiments  demonstrate  the  proposed  techniques.  The  results  can  be  used  for  sensing 
and  collaborative  autonomy,  building  base  station  coverage  maps  in  unknown  environments,  and  facilitating  multi-hop 
relaying  to  a  base  station. 
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1.  Introduction 

The  use  of  autonomous  agents  is  significantly  complicated 
by  the  need  for  wireless  networking,  with  its  challenges  of 
maintaining  scalability  and  robustness.  Radio  provides  the 
ability  to  communicate  around  and  through  barriers  in  a 
complex  environment,  but  suffers  from  rapid  signal  strength 
variation  with  relatively  small  movements  induced  by  mul- 
tipath  propagation,  as  well  as  attenuation  that  is  typically 
well  beyond  free  space  loss.  To  gain  in  robustness  and 
maintain  connectivity,  we  would  like  a  map  delineating  the 
physical  boundary  between  good  and  poor  communications 
from  some  location,  e.g.  between  a  base  station  and  its  sur- 
rounding environment.  This  is  much  more  preferable  than 
to  assume  a  fixed  distance  (disk)  model,  which  is  highly 
inaccurate  unless  the  propagation  is  essentially  free  space 
without  scattering.  Knowledge  of  this  boundary  is  useful 
in  developing  behaviors  that  maintain  reliable  communi- 
cation among  robots  or  between  robots  and  a  base  station, 
including  applications  such  as  sensor  data  collection  or  col- 
laborative control.  Furthermore,  when  considering  a  vari- 
ety of  base  stations  and  environments,  the  discovery  of  the 
boundary  between  good  and  poor  communication  scales  in 
a  manageable  way. 

Given  a  stationary  node  or  base  station,  we  can  create 
algorithms  so  that  platforms  can  maintain  communication 


with  this  base  station  with  some  relatively  high  probabil- 
ity of  success  for  each  packet  transmission.  We  assume  a 
platform  is  any  autonomous  entity  capable  of  relating  its 
position  to  a  physical  map  of  the  environment.  Determin- 
ing the  (possibly  disconnected)  physical  boundary  between 
regions  of  good  and  poor  communication  in  an  environment 
is  useful  for  constraining  the  movement  of  these  platforms. 
The  connectivity  boundary  is  selected  such  that  if  the  plat- 
form remains  inside  the  bounded  region,  then  communica- 
tions are  deemed  sufficiently  reliable.  We  refer  to  the  area  of 
good  communication  as  the  connectivity  area  with  regard  to 
the  base  station,  and  we  propose  techniques  for  building  a 
2D  map  of  this  connectivity  boundary  using  an  autonomous 
platform.  The  exploration,  mapping,  region  segmentation, 
received  signal  strength  (RSS)  sampling,  and  connectivity 
area  discovery  are  carried  out  autonomously  and  simultane- 
ously. This  requires  the  platform  to  be  capable  of  simultane- 
ous localization  and  mapping  (SLAM)  as  well  as  estimating 
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RSS  on  a  transmission  by  transmission  basis.  Our  exper- 
imental platform  is  equipped  with  a  radio  receiver,  laser 
scanner,  and  inertial  measurement  unit  (IMU)  in  order  to 
accomplish  these  tasks. 

In  a  complex  environment  with  line-of-sight  (LOS) 
and  non-line-of-sight  (NLOS)  propagation,  the  problem 
of  determining  the  good/poor  boundary  is  non-trivial. 
Exhaustive  search  and  sampling  can  be  used  to  learn  the 
propagation  environment  by  estimating  the  parameters  of 
fading  models  such  as  Rayleigh/Rician  or  Nakagami  (see, 
e.g.,  Mostofi  et  al.,  2010).  If  a  single  model  is  built,  then 
it  is  representative  of  the  environment  on  average  but,  due 
to  local  non-stationary  variation,  is  not  necessarily  good 
for  prediction.  For  example,  in  an  indoor  environment, 
the  fading  model  parameters  can  be  highly  variable  from 
room  to  room,  and  parameter  estimation  requires  many 
samples  in  each  local  area.  The  parameters  may  also 
change  with  small  local  movement  of  scatterers  such 
as  doors. 

The  problem  of  how  to  sample  and  build  a  data-driven 
model  of  a  random  field  such  as  radio  signal  propaga- 
tion in  a  non-parametric  way  is  also  challenging.  One 
approach  is  to  employ  a  kernel  to  spatially  smooth  RSS 
samples,  such  as  a  Gaussian  process.  The  use  of  a  Gaussian 
process  with  a  stationary  non-compact  kernel  function 
(Fink  and  Kumar,  2010)  presents  two  main  issues.  First, 
computational  complexity  of  the  Gaussian  process  is  not 
scalable  over  a  large  number  of  samples.  Second,  the 
Gaussian  process  does  not  provide  adequate  information 
to  guide  anything  more  than  simple  exploration  of  the 
environment. 

The  idea  of  simplified  models  and  reliance  on  mea- 
sured data  for  determination  of  wireless  connectivity  has 
been  described  in  the  robotics  literature.  Zickler  and  Veloso 
(2010)  presented  a  data-driven  model  that  relates  the  dis- 
tance between  nodes  with  the  RSS  in  a  probabilistic  way  to 
enable  tethering  and  localization  of  non-cooperative  mobile 
agents.  However,  by  relying  on  a  distance-based  abstrac- 
tion, this  method  will  reduce  to  the  standard  disk-based 
model  for  our  base  station  connectivity  area  application. 
Our  work  takes  more  inspiration  from  that  of  Hsieh  et  al. 
(2008),  where  an  outdoor  environment  is  decomposed  into 
a  number  of  cells  such  that  agents  within  the  same  cell  can 
communicate  with  high  probability. 

In  this  paper,  we  propose  and  test  algorithms  to  discover 
the  connectivity  area  for  a  base  station,  using  average  RSS 
value  and  a  threshold  to  delineate  the  goodness  of  the  chan- 
nel. The  primary  issues  are  how  to  build  the  RSS  map 
with  sufficient  information  while  not  over-parameterizing 
or  over- sampling  and  developing  exploration  algorithms 
that  efficiently  discover  the  connectivity  boundary.  We  show 
that  the  connectivity  boundary  can  be  discovered  efficiently, 
while  the  entire  connectivity  area  can  also  be  mapped  if 
desired.  With  the  complete  map,  it  is  simple  to  delineate 
regions  with  different  quality  of  connectivity  (good,  fair, 
poor,  etc.). 


2.  Problem  statement  and  approach 

We  consider  the  problem  of  autonomously  exploring  and 
mapping  a  connectivity  area  for  a  fixed  radio  transmitter 
(a  base  station).  We  define  connectivity  as  the  satisfaction 
of  a  minimum  local  average  RSS  threshold,  which  can  be 
equivalently  viewed  as  a  bit  error  rate,  capacity,  or  reli- 
ability requirement  (Proakis,  1995).  The  agent  is  initially 
deployed  within  the  base  station  connectivity  area.  Owing 
to  the  complex  propagation,  the  RSS  at  position  x  is  given 
by  the  random  field  P(x) :  ^  R.  Therefore,  the  con- 
nectivity area  can  be  determined  with  only  probabilistic 
guarantees  and  we  characterize  this  by  computing  a  local 
average  of  the  RSS  within  each  region.  With  the  base  sta- 
tion at  position  xq,  the  goal  of  the  robot  is  to  determine  the 
base  station's  connectivity  area  C  such  that  (1)  xq  g  C,  (2) 
X  e  C  ^  Fy[P(x)  >  r]  >  1  —  6,  for  6  >  0  and  some  min- 
imum RSS  r,  and  (3)  x, j  e  C  ^  x  y,  where  x  y 
indicates  the  existence  of  a  navigable  path  from  position  x 
to  position  y  that  is  contained  entirely  in  C.  Although  the 
connectivity  area  of  the  base  station  may  generally  be  dis- 
joint due  to  the  propagation  phenomena,  in  this  paper  we 
consider  only  the  region  that  contains  the  base  station.  This 
simplifies  and  limits  the  exploration  required  to  find  the 
boundary  of  C. 

Classic  fading  models,  such  as  Rayleigh/Rician  or  Nak- 
agami, can  be  employed  to  characterize  the  statistical  varia- 
tion averaged  over  an  environment.  However,  we  explicitly 
take  a  non-parametric  approach  with  an  emphasis  on  explo- 
ration and  mapping  of  the  base  station's  connectivity  area. 
A  key  idea  of  our  approach  is  to  partition  the  environ- 
ment in  a  way  that  enables  relatively  sparse  sampling  of  the 
RSS,  yet  provides  a  useful  RSS  map  and  has  sufficient  spa- 
tial resolution  to  enable  efficient  autonomous  discovery  of 
the  connectivity  area.  We  sample  locally  and  estimate  the 
RSS  mean,  as  well  as  the  local  RSS  spatial  gradient,  and 
use  the  local  gradient  to  inform  the  exploration  algorithm 
(Section  3). 

To  make  exploration  and  sampling  tractable  in  a  complex 
unknown  environment,  we  leverage  the  fact  that  radio  sig- 
nal propagation  exhibits  spatial  correlation  (Proakis,  1995; 
Goldsmith,  2005)  and  progressively  compute  a  decompo- 
sition of  the  environment  to  use  for  sampling  (Section  4). 
The  decomposition  is  computed  such  that  adjacencies  can 
be  determined  and  used  to  induce  a  graph  on  the  naviga- 
ble space  in  the  environment.  Thus,  the  exploration  problem 
can  be  cast  as  a  graph  traversal  (Section  5). 

3.  RSS  sampling  and  gradient  estimation 

Any  propagation  environment  can  be  characterized  with 
a  statistical  model,  although  this  generally  requires  many 
samples,  and  in  a  complex  environment  it  is  challenging  to 
account  for  the  spatial  non-stationarity  in  the  model  param- 
eters. A  key  idea  of  our  approach  is  to  partition  the  envi- 
ronment in  a  way  that  enables  relatively  sparse  sampling  of 
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the  RSS,  yet  provides  a  useful  RSS  map  and  has  sufficient 
spatial  resolution  to  enable  efficient  autonomous  discov- 
ery of  the  connectivity  area.  In  this  section  we  consider 
intra-region  sampling  and  estimation,  and  in  Section  4  we 
consider  partitioning  the  environment  into  regions. 

In  this  paper  we  exploit  the  RSS  estimate  provided  by 
the  receiver,  so  we  concentrate  on  estimation  and  exploita- 
tion of  the  RSS  local  spatial  mean  and  spatial  gradient. 
Measurement  of  the  RSS  has  been  analyzed  elsewhere.  For 
example,  using  a  matched  filter  detector  and  assuming  a 
Rayleigh/Rician  fading  model,  we  have  analyzed  RSS  esti- 
mation as  a  function  of  the  signal-to-noise  ratio,  the  number 
of  independent  frequency  bins,  and  LOS  (Rician)  or  NLOS 
(Rayleigh)  conditions  (Yu  and  Sadler,  2010). 

Here,  given  multiple  RSS  estimates,  we  consider  spatial 
sampling  and  estimation  of  the  average  RSS  within  a  spa- 
tial region,  as  well  as  estimation  of  the  RSS  gradient  in  that 
region.  Given  the  fading  and  physical  environment,  we  want 
a  sufficient  number  of  samples  per  region  to  obtain  reliable 
estimates  of  the  average  RSS  and  to  estimate  the  RSS  gra- 
dient which  will  inform  exploration  in  order  to  efficiently 
seek  the  base  station  connectivity  boundary. 

3.1.  Sampling  regions 

Within  each  region  we  need  a  sufficient  number  of  inde- 
pendent RSS  samples  for  estimation.  The  RSS  within  the 
region  in  a  complex  environment  may  vary  rapidly  due 
to  the  small-scale  (fast)  fading,  with  spatial  decorrelation 
affected  by  both  fast  fading  and  shadowing.  We  assume 
an  unknown  environment,  and  seek  general  guidelines  for 
sampling. 

The  RSS  average  trend  generally  obeys  decay  with 
transmit  to  receive  range  d,  where  a  varies  depending  on 
the  environment  with  2  <  a  <  5  typical.  Fast  fading  creates 
rapid  spatial  variation  around  this  average  trend. 

Consider  the  Rayleigh  fading  model  for  local  RSS 
measurements,  which  has  nominal  spatial  autocorrelation 
given  by 

R(8)=aJ^(27t8/X)  (1) 

where  8  is  the  distance  between  locations,  a  is  a  con- 
stant, Jq  is  the  zero-order  Bessel  function  of  the  first  kind, 
and  X  is  the  carrier  wavelength  (Goldsmith,  2005;  Proakis, 
1995).  This  indicates  that  samples  taken  0.3 8A,  apart  have 
zero  correlation,  and  samples  taken  at  larger  distances 
have  small  correlation.  This  heuristic  is  well-supported  in 
rich  scattering  environments  by  many  measurement  cam- 
paigns, especially  at  microwave  frequencies  (Lindhe  et  al., 
2007).  However,  the  spacing  necessary  for  decorrelation  is 
variable  depending  on  the  propagation  environment,  with 
generally  longer  correlation  in  line  of  sight  conditions.  In 
practice,  NLOS  indoor  microwave  measurements  taken  one 
or  two  wavelengths  apart  are  generally  viewed  as  experi- 
encing independent  fades.  In  other  cases,  such  as  hallways 
with  LOS,  the  distance  required  for  highly  decorrelated 


RSS  samples  can  increase  by  an  order  of  magnitude  (10- 
20  wavelengths,  say),  and  the  correlation  will  tend  to  drop 
off  less  rapidly  with  distance  (see,  e.g.,  Lindhe  et  al.,  2007). 
As  an  aside,  an  interesting  special  case  occurs  with  a  fixed 
ground  node  and  an  elevated  mobile  node;  here  the  spa- 
tial correlation  varies  slowly  in  a  trajectory  along  the  LOS 
direction,  whereas  it  varies  more  rapidly  in  a  trajectory 
orthogonal  to  the  LOS  direction  (Wadhwa  et  al.,  201 1). 

In  addition  to  spatial  variation,  independent  samples  of 
the  small-scale  fading  may  be  realized  through  frequency 
diversity,  with  the  transmitter  and  receiver  at  fixed  loca- 
tions. The  required  fi-equency  separation  is  A/  >  Be  with 
Be  ^  I /<JTm ,  where  Be  is  the  coherence  bandwidth  of  the 
channel  and  ctt^  is  the  root  mean  square  (rms)  delay  spread 
(Goldsmith,  2005).  In  indoor  channels,  rms  delay  spread 
typically  ranges  from  10  to  1000  ns  which  corresponds  to 
coherence  bandwidths  between  1  and  100  MHz  (Rappaport, 
2001).  For  the  experimental  results  presented  in  this  paper, 
we  do  not  have  control  of  center  fi-equency  and  so  do  not 
explicitly  average  over  fi-equency  to  obtain  enhanced  RSS 
estimates. 


3.2.  RSS  gradient  estimation 

The  RSS  gradient  within  a  region  yields  an  estimate 
of  the  direction  of  greatest  increase  of  RSS,  and  we 
exploit  this  as  we  characterize  the  overall  connectivity 
boundary.  Within  each  region,  we  model  the  RSS  trend  as 
linear.  To  justify  this  approximation,  consider  the  curva- 
ture of  the  exponential  path  loss  model  for  RSS,  p(d)  = 
Pod~^,  where  po  is  a  constant,  d  is  the  distance  from  the 
source  and  a  is  the  path  loss  exponent.  The  curvature  of 
a  plane  curve  p(d)  is  defined  as  /c  =  ^7(1  -\-p'^y^^ 
with p'  =       With  the  exponential  path  loss  model,  then 

K  =  a(a^l)pod-''-^/(\+aYod~^"~^f^^  ~  a(a  + 
l)pod~"~^  for  non-small  d.  Because  k  is  close  to  zero 
(note  that  a  line  has  zero  curvature),  the  linear  approxi- 
mation is  reasonable  for  regions  not  too  close  to  the  signal 
source. 

Beyond  the  path  loss,  which  varies  slowly  over  large 
distances  (100  —  1000  meters),  the  RSS  observed  at  any 
particular  location  is  also  affected  by  shadowing  and  mul- 
tipath  fading.  Shadow  effects  are  highly  correlated  over 
distances  proportional  to  the  obstructing  object  (10-100  m 
outdoors,  and  less  indoors)  (Goldsmith,  2005).  Thus,  path 
loss  and  shadowing  are  often  referred  to  as  large-scale  prop- 
agation effects  and  the  linear  approximation  of  the  gradient 
is  reasonable  when  the  sampling  region  is  relatively  small 
(e.g.  confined  to  a  region  <  10  m^),  as  is  the  case  in  our 
experiments. 

Let  P  =  {Pi,P2,...,Pn]  e  be  the  nominal  signal 
strength  at  distinct  locations  X  =  {X]^,X2, . . .  ,X^]  e 
]^7Vx2  Without  loss  of  generality,  we  shift  the  reference 
frame  to  align  the  centroid  of  X  on  the  origin  for  conve- 
nience. Using  the  linear  approximation,  the  nominal  RSS  at 
locations  X  is 
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P  =  X^  +  cl  (2) 

where  p  =  [^1,^2]^  ^  is  the  RSS  gradient  and  c  is 
a  constant  that  expresses  the  average  power  in  the  region. 
As  discussed  in  Section  3.1,  we  do  not  observe  the  nominal 
values  directly  due  to  the  multipath  fading,  but  rather  make 
measurements  P  =  {P\,P2, . . .  ,Pn]  that  fluctuate  around 
the  RSS  trend. 

The  least-squares  solution  for  ^  given  P  is 

I  =(  X^X)-^  X^P  =  X+P  (3) 

where  X+  is  the  pseudo-inverse  of  X  (Draper  and  Smith, 
1998).  Note  that  X^X  must  be  non-singular  in  order  to  solve 
for  the  gradient.  This  requires  diversity  in  two-dimensional 
motion,  e.g.  avoiding  a  straight-line  path.  Because  we 
assume  that  the  centroid  of  X  is  on  the  origin,  p  is  unaf- 
fected by  the  value  of  c  in  (2). 

It  may  be  of  interest  to  assess  the  reliability  of  y3  by  esti- 
mating Cov(  P).  For  example,  this  can  be  used  to  develop  a 
hypothesis  test  with  a  statistical  confidence  level  on  whether 
the  gradient  is  increasing  or  relatively  flat,  and  this  can  be 
input  to  the  control  algorithm  for  contour  following  (Yu  and 
Sadler,  2010).  An  approximation  to  the  confidence  level 
can  be  obtained  as  follows.  Suppose  that  P  is  unbiased. 
It  follows  that  p  is  unbiased  and  then  using  (3)  we  find 
Cov(^)=(X+)Cov(P)(X+)T.  Here  Cov(P)  depends  on 

the  distribution  of  P.  For  example,  if  each  Pi  is  indepen- 
dent and  distributed  exponentially  with  mean  Pi,  Cov(P)  is 
a  diagonal  matrix  with  Coy(  Pi,  Pi)  =  P^.  Combining  these 
results  we  obtain  an  estimate  of  Cov(  P),  which  can  be  used 
in  an  hypothesis  test  to  determine  whether  p  significantly 
deviates  fi-om  zero. 

4.  Spatial  decomposition  and  corresponding 
graph 

In  order  to  support  the  sampling  described  above,  we 
develop  a  general  method  to  decompose  the  environment 
into  regions  that  balance  several  factors.  These  factors 
include  the  geometric  complexity  and  scale  of  the  physi- 
cal environment  along  with  the  dimensions  of  the  robot, 
and  the  desire  to  have  larger  regions  and  therefore  limit 
the  number  of  regions  sampled.  Making  the  regions  small 
enough  that  they  are  generally  convex  and  result  in  a 
densely  packed  decomposition  is  an  additional  factor.  For 
a  given  decomposition,  we  compute  an  adjacency  graph, 
and  use  this  to  formulate  exploration  algorithms  that  are 
encoded  as  graph  traversals.  The  region  decomposition  and 
corresponding  graph  result  in  a  unified  approach  that  is 
applicable  across  many  environments  with  a  wide  range  of 
geometric  complexity. 

In  this  section,  we  describe  our  decomposition  algorithm 
and  the  resulting  graph.  In  Section  5,  we  couple  this  with 
autonomous  exploration,  map  building,  and  RSS  sampling. 
Consequently,  region  decomposition  is  not  carried  out  only 


once  for  the  entire  mapped  space.  Rather,  the  decompo- 
sition is  run  repeatedly  as  new  space  is  discovered,  with 
regions  that  have  been  RSS- sampled  remaining  fixed  while 
the  remaining  free  space  is  decomposed.  This  is  a  key  moti- 
vation for  our  use  of  clustering-based  algorithms  rather  than 
more  traditional  Voronoi-based  decomposition  approaches. 

Spatial  decomposition  is  a  well-studied  problem  in 
robotics  and  often  reduces  to  a  variant  of  the  Voronoi 
diagram  computation  (Choset  and  Burdick,  1995;  Thrun, 
1998;  Schwager  et  al.,  2011).  The  Voronoi  diagram  can 
be  computed  efficiently  in  realistic  settings  as  in  Lau 
et  al.  (2010),  where  the  authors  propose  an  algorithm  to 
update  a  Voronoi  diagram  based  on  input  from  a  dynamic 
occupancy-grid  representation  of  the  environment.  Owens 
and  Fields  (2010)  used  Voronoi  information  to  extract 
higher-level  semantic  information  about  the  environment 
in  order  to  identify  features  such  as  hallways,  doors,  and 
rooms.  Our  approach  draws  on  some  elements  of  these, 
using  an  occupancy  grid  supplied  by  an  underlying  SLAM 
algorithm,  but  rather  than  a  Voronoi  partition,  we  employ  a 
A:-means-based  clustering  algorithm  (Kanungo  et  al.,  2002) 
to  compute  a  decomposition  of  the  environment  where  the 
desired  cluster  size  is  a  function  of  the  geometric  complex- 
ity. This  clustering  is  used  repeatedly  during  exploration  as 
new  space  is  mapped  and  the  occupancy  grid  expands.  Once 
a  region  has  been  defined  and  the  RSS  within  that  region 
has  been  sampled,  then  that  region  is  fixed  over  fixture  map 
updates. 

Given  an  occupancy  grid  representation  of  fi*ee  space  in 
the  environment,  the  ^-means  clustering  algorithm  is  used 
to  find  a  decomposition  of  space  into  k  regions.  To  deter- 
mine the  desired  number  of  regions,  let  M  be  the  total 
number  of  open  cells  in  the  occupancy  grid  M,  and  8m  be 
the  resolution  of  the  grid  so  that  the  total  open  area  to  be 
clustered  is  of  size  M<5^  square  meters.  Then,  define  the 
characteristic  length  p  to  be  the  approximate  radius  of  a 
notional  desired  cell  size,  whose  area  is  Jtp^.  Our  target  for 
the  number  of  clusters  is  then 


For  each  iteration,  the  A:-means  algorithm  receives  the  cur- 
rent fi*ee  space  occupancy  grid  and  an  initialization  of  the  k 
region  centers.  The  ^-means  algorithm  perturbs  the  region 
centers  until  the  cells  induced  by  the  k  centers  are  approxi- 
mately the  same  size,  i.e.  with  characteristic  length  p.  This 
roughly  equates  to  finding  the  Voronoi  cells  that  cover  the 
free  space,  i.e.  each  region  contains  the  set  of  points  that 
are  closest  to  its  center.  Choice  of  p  is  governed  by  the  geo- 
metric feature  size  in  the  environment,  the  size  of  the  robot, 
and  the  radio  wavelength.  In  our  experiments  p  is  fixed.  We 
discuss  the  choice  of  p  below. 

4.1.  Decomposition  algorithm 

Our  decomposition  algorithm  is  summarized  in  Algo- 
rithm 1  with  subroutine  Algorithm  2.  At  any  given  time. 
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Algorithm  1  Decompose  Environment 

Require:  Characteristic  length  p,  free  space  cells  mt  e  M, 
unsampled  region  centers  rj^^^ 
I:  k  ^  MSl^/np^  {Compute  number  of  regions} 
2:  M  =  M  {Unassigned  free  cells  for  new  regions} 
3:  whiles  >  ||7^||  do 

4:     0  ^  UniformRandom(  A^)  fory  =  1 ...  A:  {initial- 
ize new  region  centers} 
5:  ^  A:-means(  M,rj,  rj^^^)  algorithm 

6:     for  all  7^y  do 

7:  if  drj-rui  —  P 

8:        n  =  nunj 

9:  else 

10:  nii  e  IZj  are  removed  from  M 

1 1 :  end  if 

12:  end  for 

13:  k^MSl/np^ 

14:  end  while 

15:  T^y  ^  Ensure  Region  Continuity ( T^y) 

16:  return  IZ  {Set  of  sampling  regions} 


the  decomposition  is  carried  out  over  the  currently  known 
free  space,  characterized  by  the  occupancy  grid  obtained 
from  the  underlying  SLAM  algorithm.  A  region  is  char- 
acterized by  its  centroid  ry,  and  the  elements  of  the  occu- 
pancy grid  assigned  to  that  cluster.  We  refer  to  two  types  of 
regions:  sampled  and  unsampled.  A  sampled  region  refers 
to  a  region  that  the  robot  has  entered  and  collected  enough 
RSS  samples.  We  refer  to  all  other  regions  as  unsampled, 
where  the  total  number  of  samples  is  below  the  sampling 
threshold. 

As  exploration  and  sampling  proceeds,  the  sampled 
regions  are  permanently  fixed.  Thus,  the  input  to  the  decom- 
position algorithm  includes  the  currently  known  free  space 
occupancy  grid  which  is  not  assigned  to  sampled  regions, 
and  an  initialization  of  the  region  centers.  Some  of  the 
region  centers  initializing  the  ^-means  algorithm  are  from 
unsampled  regions  created  in  the  previous  decomposition, 
which  we  refer  to  as  rj^^^.  After  a  clustering  and  sampling 
step,  the  robot  will  also  have  augmented  the  known  free 
space.  Thus,  each  time  the  clustering  algorithm  is  executed, 
it  is  given  an  initial  set  of  centers,  composed  of  the  old 
unsampled  region  centroids  ry^^^,  and  a  uniformly  random 
set  Vj  taken  over  the  remaining  free  space.  We  emphasize 
that  the  clustering  algorithm  uses  rj^^^  for  initialization, 
but  these  will  generally  shift  as  the  clustering  algorithm 
accommodates  the  newly  discovered  free  space. 

The  A:-means  algorithm  (line  5  of  Algorithm  1)  returns 
a  candidate  set  of  clusters,  denoted  by  IZj,  j  =  I . .  .k, 
attempting  to  decompose  into  k  regions  with  area  roughly 
equal  to  Ttp^.  Because  the  clustering  is  carried  out  as 
exploration,  mapping,  and  RSS-sampling  proceeds,  the 
occupancy  grid  may  be  disconnected  (non-contiguous). 
To  account  for  this,  we  then  check  the  regions  IZj  for 
appropriate  size  and  shape. 


Algorithm  2  Ensure  Region  Continuity 
Require:  regions  71 

1:  for  all7^/  G  7^do 

2:     for  all  Pi  e  IZj  do 

3:  if  Pi  has  no  neighbors  labeled  as  same  region  then 
4:  =      \Pi  {Remove pi  from  Tlj} 

5:        end  if 
6:     end  for 
7:  end  for 

8:  return  IZj  {return  the  points  that  constitute  a  continu- 
ous region} 


Algorithm  1,  line  7,  compares  the  average  point-to- 
centroid  distance  to  a  threshold  for  each  region.  We  reject 
any  region  whose  average  point-to-centroid  distance  is 
greater  than  p.  This  pruning  strategy  favors  regions  that 
are  roughly  circular  and  results  in  densely  packed  regions 
that  are  typically  convex,  as  shown  in  our  experiments 
subsequently.  Once  identified,  grid  cells  that  form  poorly 
shaped  regions  are  removed  from  the  list  of  unsampled 
open  grid  cells  in  fine  10.  The  loop  will  execute  until 
all  of  the  regions  created  by  A:-means  meet  the  average 
centroid-to-point  distance  constraint. 

A  second  check  is  then  carried  out  via  the  call  to  Algo- 
rithm 2  in  line  16.  This  simple  filtering  step  removes  sin- 
gle occupancy  grid  cells  that  are  disconnected  from  their 
assigned  region,  i.e.  they  have  no  neighbors  within  their 
assigned  region.  We  find  experimentally  that  disconnected 
grid  cells  can  occur,  for  example,  near  barriers.  Any  grid 
cells  pruned  from  the  current  clustering  will  be  retained  in 
the  free  space  occupancy  grid  for  subsequent  clustering  as 
the  robot  continues  its  exploration. 

At  completion.  Algorithm  1  returns  the  region 
decomposition  in  the  set  IZ.  We  then  analyze  the  region 
decomposition  to  compute  the  adjacency  graph  for  this 
decomposition  of  the  known  environment.  Adjacency 
between  regions  is  based  on  the  ability  of  the  robot  to  navi- 
gate from  one  region  to  the  other,  where  we  assume  regions 
are  adjacent  if  the  grid  cells  of  two  regions  neighbor  each 
other.  This  is  informed  by  our  choice  of  p.  If  regions  are 
too  large,  then  regions  created  near  narrow  spaces  might 
not  have  grid  cells  neighboring  each  other.  Thus,  while 
the  characteristic  region  length  p  should  be  large  enough 
to  allow  independent  RSS  samples  (i.e.  sufficient  spatial 
separation),  it  also  needs  to  be  small  enough  to  allow  for 
regions  to  be  roughly  convex. 

The  exploration  algorithm  will  use  the  adjacency  graph 
to  inform  its  next  mobility  step,  and  the  entire  process  is 
repeated  until  the  base  station  connectivity  area  has  been 
mapped. 

5.  Exploration  algorithms 

To  map  out  the  base  station  connectivity  area  C  in  an  effi- 
cient way,  we  combine  exploration,  RSS  sampling,  RSS 
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gradient  estimation,  and  the  process  of  graph  generation 
described  in  Section  4.  We  compare  exhaustive  exploration 
with  a  connectivity  contour  seeking  approach.  The  exhaus- 
tive approach  yields  a  complete  decomposition  of  C  with 
RSS  measured  in  each  region,  whereas  the  contour  seeking 
method  yields  a  decomposition  of  C  that  ideally  encom- 
passes only  the  connectivity  boundary.  If  only  the  boundary 
is  desired,  the  contour  seeking  method  can  be  much  more 
efficient  than  exhaustive  mapping. 

Exploration  proceeds  by  updating  the  environment  map 
in  the  form  of  the  occupancy  grid,  a  decomposition  is 
carried  out  over  the  current  free  space  and  an  updated 
graph  is  computed  as  described  in  Section  4,  and  then 
the  next  region  for  RSS  sampling  is  selected.  The  robot 
moves  to  this  region  and  performs  RSS  sampling,  and  the 
process  is  repeated  until  the  entire  base  station  connec- 
tivity area  is  mapped.  The  decomposition  occurs  after  the 
robot  has  finished  RSS  sampling  a  region  and  updating  the 
map.  Regions  that  have  been  sampled  are  kept  because  we 
assume  that  the  environment  is  static.  Thus,  the  robot  cre- 
ates new  regions  incrementally,  with  the  criterion  for  choice 
of  the  next  region  to  RSS  sample  dictated  by  the  overall 
exploration  goal. 

5.7.  Exhaustive  exploration 

An  exhaustive  exploration  algorithm  requires  that  the  robot 
visit  and  sample  every  region  in  the  component  of  the  graph 
connected  to  the  base  station.  This  essentially  constitutes 
a  solution  to  the  Traveling  Salesman  Problem.  However, 
because  a  map  of  the  entire  environment  is  not  available 
a  priori,  we  instead  pursue  a  greedy  heuristic. 

The  algorithm  is  summarized  in  Algorithm  3.  Given  a 
partial  map  of  the  environment,  a  decomposition  is  com- 
puted to  yield  sample  regions  7Z,  and  its  adjacency  graph 
is  computed.  Exploration  proceeds  by  finding  the  closest 
unsampled  region  in  71  that  is  connected  to  the  base  sta- 
tion through  the  adjacency  graph.  The  robot  drives  to  this 
region,  updates  the  SLAM  algorithm,  and  makes  RSS  mea- 
surements. A  new  decomposition  is  computed  based  on 
the  updated  occupancy  grid,  minus  the  regions  that  are 
sampled  which  are  permanently  fixed  for  the  experiment. 
This  process  continues  until  there  are  no  unsampled  regions 
connected  to  the  base  station. 

5.2.  Contour-seeking  exploration 

The  goal  of  the  contour-seeking  exploration  algorithm, 
summarized  in  Algorithm  4,  is  to  efficiently  drive  to  and 
sample  RSS  only  along  the  outer  contour  of  the  base  sta- 
tion's connectivity  area  C.  We  enable  this  behavior  by  using 
the  RSS  gradient  to  predict  RSS  averages  inside  unsampled 
regions  that  are  adjacent  to  the  sampled  region  currently 
occupied  by  the  robot.  After  sampling  a  region  IZj,  the  robot 
chooses  the  next  region  to  sample  IZnext  based  on  which  of 
the  adjacent  regions  has  the  lowest  predicted  RSS.  The  RSS 


Algorithm  3  Exhaustive  Exploration 
Require:  Initial  map  M 

\  :  71  ^  Decompose  Environment(  M) 

2:  Compute  adjacency  graph 

3:  K  ^  Unsampled  regions  connected  to  base  station 

4:  while  3  7lj  eii  do 

5:  Find  closest  unsampled  region  7Zciosest  ^  ^ 

6:  Drive  to  TZdosest  and  collect  RSS  samples 

7:  if  Average  RSS  in  TZdosest  <  then 

8:  Disconnect  TZdosest 

9:  end  if 

10:  7^  ^  DecomposeEnvironment( 

1 1 :  Compute  adjacency  graph 

12:  TZ  ^  Unsampled  regions  connected  to  base  station 

13:  end  while 


prediction  assumes  that  the  negative  of  the  gradient  is  the 
direction  of  steepest  descent  and  therefore  points  toward 
the  neighboring  region  with  the  lowest  RSS.  If  all  adja- 
cent regions  have  been  sampled  already,  the  robot  moves 
to  the  region  with  the  lowest  RSS  that  is  still  inside  the 
connectivity  area  of  the  base  station  and  continues. 

Upon  initialization  near  the  base  station,  the  robot  will 
move  away  from  the  base  station  along  a  line  of  estimated 
steepest  RSS  descent  until  it  encounters  a  region  that  is 
below  the  connectivity  threshold  x  dB.  With  this  crossing 
of  the  connectivity  threshold,  the  robot  has  encountered  the 
base  station  connectivity  boundary.  When  the  connectivity 
boundary  is  crossed,  the  gradient  is  used  to  predict  a  neigh- 
boring region  whose  RSS  will  be  highest,  i.e.  will  ideally 
lead  to  a  region  inside  the  connectivity  area.  Proceeding 
in  this  manner,  the  connectivity  boundary  will  be  discov- 
ered and  regions  adjacent  to  the  connectivity  boundary  will 
be  mapped  as  the  robot  zig-zags  along  this  boundary.  As 
the  robot  proceeds  to  discover  the  connectivity  boundary, 
interior  regions  with  RSS  above  the  threshold  r  may  be 
pruned  from  the  graph.  The  remaining  leaf  nodes  define  the 
connectivity  boundary. 

The  contour  seeking  control  strategy  results  in  fewer 
total  regions  sampled  than  the  exhaustive  approach.  As  with 
the  exhaustive  search,  only  regions  connected  to  the  initial 
region  are  considered  for  sampling.  With  the  contour  seek- 
ing method  some  areas  within  the  connectivity  area  will  not 
be  identified  for  sampling.  We  assume  that  the  area  within 
the  boundary  has  good  connectivity,  while  noting  that  in 
some  complex  environments  this  might  not  be  strictly  true. 

6.  Experimental  configuration 

Our  experiments  were  performed  using  a  Packbot,  shown  in 
Figure  1,  equipped  with  a  skid-steer  drive  system,  and  on 
board  computation.  Both  IEEE  802. 1 1  and  Zigbee  radios 
were  on  the  platform,  and  the  Zigbee  provided  the  RSS 
measurements.  The  Zigbee  operated  at  2.4  GHz  with  cor- 
responding wavelength  of  12.5  cm.  The  Packbot  was  also 
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Algorithm  4  Contour  Seeking  Exploration 
Require:  Initial  map  M 

I:  71  ^  Decompose  Environment(  A^) 

2:  Compute  adjacency  graph 

3:  iZ  ^  Unsampled  regions  connected  to  base  station 

4:  ^  Unsampled  perimeter  regions 

5:  while  3  TZj  e  W  do 

6:  if  31Z  Adjacent  to  IZnext  then 

7:  Tlnext        ^     lowcst    predicted    region  in 

Adjacent(  TZiast) 

8:  else 

9:  Tlnext  ^  lowcst  predicted  region  in  iZ 

10:  end  if 

1 1 :  Drive  to  IZnext  and  collect  RSS  samples 

12:  if  Average  RSS  in  IZnext  <  then 

13:  Disconnect  7^„e;c^ 

14:  end  if 

15:  7^  ^  DecomposeEnvironment(  Al) 

1 6 :  Compute  adj acency  graph 

17:  IZiast  ^  TZnext 

18:  IZ  ^  Unsampled  regions  connected  to  base  station 

19:  Vf  ^  Unsampled  perimeter  regions 

20:  end  while 


equipped  with  a  Hokuyo  UTM-30LX  scanning  laser  range 
finder  with  30  m  range  and  Microstrain  3DM-GX2  IMU, 
utilized  by  the  SLAM  algorithm.  A  Zigbee  transmitter  was 
used  as  a  fixed  base  station,  periodically  broadcasting  a 
beacon  at  12  Hz,  so  this  defines  the  maximum  RSS  mea- 
surement rate.  Note  that  at  lower  signal-to-noise  ratio  some 
beacon  packets  may  be  undetected  and  therefore  some  RSS 
measurements  lost.  For  each  successfully  received  beacon 
packet  we  recorded  the  current  position  (as  estimated  via 
SLAM)  and  the  reported  RSS. 

Autonomous  SLAM  and  navigation  were  implemented 
by  leveraging  libraries  running  on  the  open-source  Robotics 
Operating  System  (ROS).  These  libraries  enabled  the  robot 
to  dynamically  build  a  map  and  navigate  throughout  an  ini- 
tially unknown  2D  environment.  In  each  experiment  the 
robot  was  initially  deployed  near  the  base  station,  without 
any  prior  map  or  knowledge  of  the  environment  (unless  oth- 
erwise specified).  Thus,  the  exploration,  mapping,  region 
segmentation,  RSS  sampling,  and  connectivity  area  discov- 
ery were  carried  out  autonomously  and  simultaneously. 


6.1.  Experimental  parameters 

The  parameters  in  Table  1  are  consistent  across  all  experi- 
ments. The  ZigBee  radio  reported  RSS  measurements  peri- 
odically at  12  Hz.  If  no  measurement  was  available,  e.g. 
because  of  dropped  packets,  this  was  also  reported.  At  each 
measurement  time  the  robot  position  and  RSS  measurement 
were  recorded.  The  map  resolution  5^  was  determined  by 
the  choice  of  ROS-based  SLAM  algorithm. 


Fig.  1.  The  Packbot  experimental  platform,  including  laser  scan- 
ner and  IMU  for  SLAM,  and  Zigbee  radio  for  RSS  measurements. 

Table  1.  Table  of  parameters. 


Parameter  Value 


RSS  sample  rate 

12  Hz 

Map  resolution  {8m) 

0.25  m 

Characteristic  region  length  (p) 

0.8  m 

Robot  length 

0.7  m 

Robot  width 

0.4  m 

Receiver  placement 

0.23  m  off-center 

Connectivity  threshold  (r) 

Variable  (dB) 

RSS  sampling  threshold 

Variable 

The  characteristic  length  p  is  a  critical  parameter  that 
specifies  the  desired  region  size  for  spatial  decomposition. 
Choice  of  p  is  influenced  by  the  geometric  complexity  of 
the  environment,  and  the  size  and  maneuverability  of  the 
platform.  Experimental  decompositions  with  different  val- 
ues of  p  showed  that  p  =  0.8  m  was  a  good  choice  because 
the  resultant  regions  are  large  enough  for  good  separation 
of  RSS  measurements  while  small  enough  to  allow  the 
robot  to  form  regions  even  in  confined  areas.  For  exam- 
ple, if  p  is  too  large  then,  regions  may  take  non-convex 
and  irregular  shapes;  this  may  be  acceptable  in  a  given 
application,  although  it  somewhat  complicates  the  adja- 
cency graph  definition  and  trajectory  planning.  We  chose 
p  =  0.8  m,  which  resulted  in  region  decompositions  whose 
region  shapes  were  generally  convex  despite  the  geometric 
complexity  of  various  test  environments. 

The  Packbot  has  a  relatively  large  0.7  mx  0.4  m  footprint. 
To  facilitate  RSS  sampling  in  a  confined  space,  we  mounted 
the  Zigbee  radio  antenna  0.23  m  in  front  of  the  robot's  center 
of  rotation.  A  simple  rotation  in  place  then  yielded  a  set 
of  spatially  diverse  RSS  samples,  and  this  maneuver  was 
executed  within  each  region  (see  Figure  8). 

The  base  station  connectivity  area  is  defined  by  an  RSS 
threshold  r,  that  specifies  the  acceptable  quality  of  a  com- 
munications link,  and  depends  on  the  transmit  power  and 
propagation  loss.  Lower  z  implies  that  more  propagation 
loss  is  acceptable,  and  results  in  a  larger  connectivity  area 


113 


for  a  fixed  base  station  transmit  power.  We  experimented 
with  different  values  for  r  as  described  below. 

7.  Connectivity  area  mapping  experiments 

In  this  section  we  describe  two  connectivity  area  map- 
ping experiments,  outdoors  in  an  uncluttered  courtyard,  and 
indoors  with  obstacles. 

7.7.  Open  outdoor  environment 

Experiments  were  conducted  in  an  open  courtyard,  as  illus- 
trated in  Figures  2a,  2b,  and  3.  The  black  areas  along 
the  right  and  top  are  low  concrete  partitions  (obstacles)  in 
the  courtyard  of  a  multi-story  building.  Figure  2a  shows  the 
region  decomposition  result  for  one  run  of  the  exhaustive 
exploration  algorithm.  To  test  the  decomposition  algorithm 
in  a  controlled  manner,  the  map  was  generated  a  priori  for 
this  experiment.  The  regions  are  roughly  uniformly  shaped 
in  this  uncluttered  area.  Some  small  spatial  gaps  occur  that 
are  not  interior  to  any  region  and  are  much  less  than  the 
characteristic  length  p.  The  corresponding  adjacency  graph 
is  shown  in  Figure  2b.  The  region  centroids  are  plotted  as 
squares,  with  lines  indicating  adjacent  regions. 

The  contour  seeking  Algorithm  4  was  tested  in  the  court- 
yard, with  the  RSS  threshold  t  set  such  that  the  connec- 
tivity area  was  confined  within  the  size  of  the  courtyard. 
An  experimental  run  is  depicted  in  Figure  3  where  base 
station  signal  source  is  indicated  by  a  dot  and  the  result- 
ing contour  is  shown  as  a  dashed  line.  The  average  RSS 
per  region  is  mapped  to  the  gray  scale  shown.  The  robot 
is  initially  placed  near  the  base  station  signal  source. 
Once  the  robot  has  sampled  a  region,  it  follows  the  gra- 
dient (denoted  by  a  outlined  arrow)  to  the  region  of  low- 
est estimated  average  RSS.  It  continues  to  do  this  until 
it  samples  a  region  with  RSS  lower  than  r,  which  first 
occurs  at  about  location  ( —4, 1)  m.  Then,  the  robot  retreats 
(denoted  by  a  solid  arrow)  to  an  unexplored  region  which 
is  estimated  to  have  average  RSS  above  r.  The  robot  then 
works  its  way  around  the  environment,  alternating  between 
advance-retreat  across  the  boundary,  mapping  the  connec- 
tivity boundary  (shown  as  a  dashed  cyan  line).  As  shown 
in  the  figure,  roughly  30%  of  the  courtyard  area  remains 
unsampled,  illustrating  the  efficiency  gain  using  the  con- 
tour following  approach.  The  efficiency  gain  versus  exhaus- 
tive exploration  will  grow  as  the  overall  connectivity  area 
becomes  larger,  e.g.  with  a  higher  power  base  station. 

7.2.  Complex  indoor  environment 

Experiments  were  carried  out  in  several  different  indoor 
environments.  A  challenging  example  is  shown  in  Figure  4, 
conducted  inside  a  fire  house.  This  room  had  concrete  block 
walls,  and  four  large  metal  garage  doors.  There  were  also 
obstacles  including  metal  lockers  in  the  center  of  the  room 
(shown  in  black),  and  a  large  table  along  one  wall.  Thus,  the 
environment  presented  both  geometric  challenges  as  well  as 
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Fig.  2.  Outdoor  courtyard  experimental  decomposition  results, 
(a)  Sampling  region  boundaries,  (b)  The  corresponding  adjacency 
graph.  Squares  represent  nodes  and  lines  represent  adjacencies. 

complex  radio  propagation.  For  example,  note  that  there  is 
relatively  high  RSS  even  in  some  non-line  of  sight  regions 
due  to  the  radio  reflectivity  from  the  large  amount  of  metal 
in  the  environment. 

A  example  run  of  the  contour  seeking  algorithm  is 
depicted  in  Figure  4.  Note  that  in  this  experiment,  the 
robot  does  not  smoothly  track  the  trajectory  near  (—6, 2)  m. 
However,  the  algorithm  is  robust.  The  robot  successfully 
maps  out  the  connectivity  area,  despite  noisy  gradient  esti- 
mates and  obstacles  intersecting  the  connectivity  boundary. 


8.  Varying  the  RSS  threshold 

The  connectivity  area  is  a  function  of  the  RSS  threshold 
T,  with  lower  r  resulting  in  an  extended  area.  We  illustrate 
this  using  two  threshold  values  in  the  same  environment. 
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Fig.  3.  Outdoor  courtyard  experiment  using  the  contour  seeking 
Algorithm  4.  Obstacles  are  black  in  this  overhead  view  of  the 
environment.  The  dashed  line  indicates  the  connectivity  bound- 
ary, with  arrows  indicating  estimates  of  the  RSS  gradient  as  the 
exploration  evolves.  Average  RSS  per  region  is  mapped  to  gray 
scale. 
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Fig.  4.  Indoor  fire  house  experiment.  The  contour  seeking  algo- 
rithm was  used  to  find  the  connectivity  area  in  this  complex 
environment. 
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Fig.  5.  Indoor  connectivity  areas  mapped  with  the  contour  seeking  algorithm,  with  different  values  for  RSS  threshold  r.  (a)  Higher 
RSS  threshold  results  in  a  smaller  connectivity  area,  (b)  Lower  RSS  threshold  results  in  a  larger  connectivity  area,  including  LOS  and 
NLOS  regions. 


a  concrete  block  building  with  window  spaces,  and  obsta- 
cles including  wooden  desks  and  a  large  PVC  pipe.  Two 
experiments  are  shown  in  Figure  5a  and  (b). 

In  Figure  5(a),  r  =  —73  dB.  Using  the  contour  following 
algorithm,  the  robot  maps  a  connectivity  area  that  is  almost 
exclusively  in  LOS  of  the  base  station  transmitter.  In  Figure 
5b  r  is  lowered  to  —  79  dB.  As  a  result  the  connectivity  area 


is  larger  and  encompasses  line  of  sight  and  non-line  of  sight 
regions. 

9.  Base  station  siting 

Exhaustive  exploration  provides  a  complete  picture  of  the 
connectivity  area,  with  each  region  characterized  by  its 
average  RSS.  Experimental  results  are  shown  in  Figure  6a 


115 


Fig.  6.  Indoor  connectivity  area  maps  for  two  different  base  sta- 
tion sites,  (a)  Base  station  sited  at  ( —0.25, 0.35).  (b)  Base  station 
sited  at  (1.8,4.1). 


and  (b)  in  a  large  office  building  lobby.  The  room  shape 
is  somewhat  irregular  as  indicated  by  the  white  area  with 
walls  in  black.  There  are  reinforced  concrete  walls,  exterior 
walls  with  windows,  and  irregular  ceiling  height.  Chairs, 
tables,  and  structural  columns  serve  as  obstacles  for  the 
robot.  Overall,  this  environment  has  very  complex  radio 
propagation. 

The  exhaustive  exploration  Algorithm  3  was  used  for 
two  different  base  station  source  locations  as  shown  in 
the  figures.  To  illustrate  the  complexity  of  the  environ- 
ment, an  RSS  threshold  contour  is  drawn  with  threshold  r 
set  to  —  60  dB.  This  threshold  was  chosen  to  highlight  the 
irregular  connectivity  area  that  results,  including  shadowing 


effects  due  to  obstacles.  The  contours  are  not  obvious  from 
the  room  geometry.  Note  how  the  average  estimated  RSS 
in  some  regions  is  below  the  threshold,  even  when  they  are 
rather  close  to  the  base  station  and  have  LOS  propagation. 

Also  shown  in  Figure  6a  and  (b)  are  circles  showing  con- 
servative estimates  of  the  connectivity  boundary  based  on  a 
simple  disk  model  for  the  radio  propagation.  A  path  loss 
model  was  fitted  to  the  data  collected  during  the  experi- 
ment shown  in  Figure  6a.  In  this  figure  the  radius  d  that 
solves  X  =  Lq  —  lOalogjo^  represents  a  conservative 
estimate  of  the  connectivity  boundary,  where  Lq  =  —50 
dB  and  a  =  2.  Assuming  that  the  path  loss  character- 
izes the  environment,  the  same  disk  is  plotted  in  Figure  6b. 
Although  these  are  based  on  averages  taken  within  the  envi- 
ronment, the  disk  represents  a  weak  prediction  of  the  actual 
connectivity  area. 

10.  RSS  sampling 

To  obtain  a  good  estimate  of  the  RSS  and  its  gradient 
within  a  region,  the  robot  should  take  sufficiently  many 
spatially  separated  measurements,  as  discussed  in  Sec- 
tion 3.2.  In  practice  it  is  not  good  to  assume  that  either 
the  samples  are  sufficiently  spaced  as  to  be  uncorrected 
or  that  the  spatial  correlation  is  known  a  priori.  In  our 
scenario,  neither  assumption  would  necessarily  hold  for 
many  reasons  including  imperfect  odometry  (motion  slip- 
page, non-constant  velocity,  etc.)  and  insufficient  sample 
spacing  distances  between  consecutive  samples.  However, 
we  can  use  the  sample  variance  of  the  RSS  measurements 
to  determine  when  a  region  has  been  sufficiently  sampled. 

To  study  the  required  sample  size  experimentally,  con- 
sider a  collection  of  N  RSS  measurements  Pj,j  =  I,. . .  ,N, 
taken  within  a  region.  The  size  of  N  depends  on  the  sam- 
pling rate  (nominally  12  Hz  in  our  experiments),  the  explo- 
ration path  of  the  robot  within  the  region,  and  the  number 
of  dropped  packets.  In  the  results  we  present  in  this  sec- 
tion, we  only  use  measurements  that  were  made  while  the 
robot  was  moving  to  improve  the  spatial  diversity  of  the 
samples.  Although  the  spatial  decomposition  yields  regions 
of  roughly  the  same  size,  the  robot  spends  varying  amounts 
of  time  within  each  region  and  therefore  the  number  of  sam- 
ples per  region  are  not  identical.  Figure  7  shows  the  samples 
per  region,  and  the  corresponding  spatial  distance  between 
consecutive  samples,  for  the  indoor  experiment  depicted  in 
Figure  8.  The  robot  makes  around  50  to  200  RSS  samples 
per  region.  The  consecutive  sample  spacing  is  typically  less 
than  half  a  wavelength  (6.25  cm  at  2.4  GHz)  because  the 
12  Hz  RSS  measurement  rate  is  high  relative  to  the  robot 
speed.  However,  as  we  show  below,  these  samples  are  suffi- 
cient in  number  and  spacing  for  good  estimation  of  the  RSS 
average  in  each  region. 

Figure  8  shows  the  robot  trajectory,  where  RSS  sam- 
ple sites  within  the  same  region  are  plotted  with  the  same 
color.  The  robot  rotation  for  spatial  sampling  is  evident  in 
the  looping  trajectory.  We  have  previously  presented  this 


116 


0  50        100       150       200       250  0       0.05     0.1      0.15     0.2  0.25 

Samples  per  Region  Distance  between  Samples  (m) 


Fig.  7.  Left:  Histogram  of  the  number  of  RSS  samples  taken  per  region  in  an  indoor  environment  (cf.  Figures  4  and  8).  Right:  Histogram 
of  consecutive  sample  spacing,  indicating  that  many  samples  have  spacing  less  than  one  wavelength. 


Trajectory  of  Robot 


Distance  (m) 

Fig.  8.  Exploration  path  with  RSS  sampling  locations  for  an  indoor  fire  house  environment  (cf.  Figure  4).  Samples  within  a  region  have 
the  same  color.  The  loops  are  samples  taken  during  robot  rotation. 


experiment  in  Section  7.2,  Figure  4.  Consider  the  samples 
taken  in  region  17  and  region  28.  The  sampling  of  region 
17  yields  N  =  111  measurements  while  that  of  region 
28  yields  N  =  114.  Although  the  values  for  N  are  close, 
the  sampling  trajectories  are  quite  different.  Recalling  from 
Section  3.2  that  X  is  the  centered  matrix  of  sampling  loca- 
tions, then  a  measure  of  the  spread  of  the  samples  within  a 
region  is  A  =  ^^1^2,  where  A/  is  the  i^^  eigenvalue  of  the 
2x2  matrix  X^X.  The  greater  this  metric,  the  more  spread 
out  the  RSS  samples  are  within  the  region.  Here,  region  17 


yields  ( A  =  0.46)  whereas  region  28  yields  ( A  =  0.38), 
indicating  that  region  17  has  slightly  better  spatial  spread  in 
the  RSS  sampling  locations.  Increased  spatial  separation  of 
the  RSS  samples  improves  the  RSS  and  gradient  estimate. 

The  top  panels  of  Figure  9  depicts  the  sampling  trajec- 
tory for  regions  17  and  28.  Time  evolution  is  depicted  by 
color  transformation  from  dark  (earlier  samples)  to  light 
(later  samples).  The  middle  panels  plot  the  RSS  samples 
that  showing  the  significant  small  scale  variation  in  RSS 
with  mobility.  Also  on  the  middle  two  panels  we  overlay 
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Fig.  9.  Detailed  look  at  two  regions  in  an  indoor  environment  (cf.  Figure  4).  Top  row:  exploration  path.  Middle  row:  RSS  measurements 
and  evolution  of  RSS  estimate.  Bottom  row:  Evolution  of  the  variance  of  the  RSS  mean. 


the  evolution  of  the  RSS  mean  (solid  line)  as  the  sample 
size  grows,  calculated  by  Pj  =  j  J2]=i  A-  The  evolving 
RSS  mean  estimate  is  bracketed  by  a  one  standard  deviation 
interval  (dashed  lines)  given  by      =  Var(Py)  =  j  Vary(P). 

The  unbiased  sample  variance  is  YaVj(P)  ~  5Z^=i(  A  — 
Pj)^.  This  approximation  does  not  consider  the  effect  of 
correlated  samples. 

The  bottom  two  panels  in  Figure  9  plot  the  evolving  sam- 
ple variance  of  the  RSS  mean  estimate.  In  region  17  the 
variance  drops  quickly,  indicating  high  confidence  in  the 
RSS  mean  estimate  as  the  sample  size  grows.  In  region  28, 
the  variance  of  the  RSS  mean  has  more  interesting  behavior. 
It  drops  rapidly  and  then  rises  sharply  again  before  finally 
decaying.  Looking  at  the  map  in  Figure  8,  we  see  that  the 
robot  mobility  introduces  an  additional  barrier  between  it 
and  the  base  station  around  sample  number  20.  The  addi- 
tional barrier  results  in  a  sharp  drop  in  the  RSS,  with  a 
corresponding  sharp  rise  in  the  variance  of  the  RSS  mean 
within  the  region.  However,  as  the  robot  continues  to  move 
and  further  measurements  are  made  the  variance  dies  down 
again,  leading  to  a  good  estimate  of  the  RSS  mean  for  that 
region. 

Figure  10  summarizes  the  variance  of  the  RSS  estimate 
YdLv(Pj)  for  the  entire  experiment.  The  left  panel  shows  the 
variance  for  each  region,  plotted  against  the  above  metric 


for  spatial  separation.  It  is  clear  that  increased  spatial  spread 
in  the  data  results  in  lower  variance.  The  right  panel  depicts 
the  variance  for  each  region  as  a  function  of  the  total  num- 
ber of  RSS  samples  collected,  showing  a  clear  preference 
for  more  samples.  Note  that  the  spatial  spread  can  be  cal- 
culated online,  so  that  the  exploration  algorithm  could  be 
modified  to  continue  intra-region  sampling  until  a  desired 
spatial  spread  and/or  total  sample  size  is  achieved. 


11.  Conclusions 

We  have  developed  and  demonstrated  new  algorithms  for 
mapping  the  physical  boundary  of  a  radio  connectivity  area 
within  a  map.  Our  approach  uses  region  decomposition 
and  sampling  to  estimate  the  RSS  average  in  unexplored 
regions.  Heuristics  for  computing  good  regions  and  RSS 
sampling  were  postulated.  These  heuristics  were  applied 
to  develop  a  region  decomposition  algorithm.  We  demon- 
strated contour  seeking  and  exhaustive  exploration  algo- 
rithms that  exhibit  robustness  to  environmental  complexity. 
The  resulting  control  algorithms  are  applicable  to  a  wide 
variety  of  scenarios  involving  a  base  station  as  they  do  not 
require  knowledge  of  the  propagation  environment  beyond 
basic  fading  assumptions  and  do  not  require  a  map  a  priori. 
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Fig.  10.  Variance  of  RSS  mean  estimate  decreases  as  spatial  sepa- 
ration ( A)  increases  (left)  and  as  the  number  of  samples  increases 
(right). 

Our  methods  naturally  fit  into  and  extend  autonomous 
exploration  and  mapping  to  characterize  network  connec- 
tivity. Some  avenues  for  further  research  include  coupling 
RSS  with  a  particular  physical  layer  radio  technology,  e.g. 
coupling  with  packet  loss  probability  or  measures  of  quality 
of  service.  Larger-scale  outdoor  experiments  are  of  inter- 
est, e.g.  in  establishing  coverage  areas  for  cellular  telephone 
systems  that  are  rapidly  deployed  in  an  emergency.  Also, 
our  method  could  be  extended  to  a  multiple  base  station 
system.  We  have  assumed  a  constant  transmit  power  at  the 
base  station,  so  an  extension  to  incorporate  power  con- 
trol could  couple  quality  of  service  guarantees  as  agents 
move  in  the  connectivity  region,  or  to  service  agents  dur- 
ing base  station  hand  off  as  they  traverse  from  one  cell  to 
another.  More  generally,  base  station  technology  such  as 
antenna  arrays  and  frequency  diversity  could  also  be  incor- 
porated. We  have  shown  how  the  number  of  RSS  samples 
needed  can  be  estimated  during  collection,  and  this  could  be 
incorporated  into  the  exploration  and  decomposition  algo- 
rithms. Finally,  we  note  that  in  different  physical  environ- 
ments it  could  be  advantageous  to  spatially  adapt  the  size  of 
sampling  regions,  p. 
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Abstract — The  purpose  of  this  paper  is  to  review  research  per- 
taining to  the  Hmitations  and  advantages  of  supervisory  control  for 
unmanned  systems.  We  identify  and  discuss  results  showing  tech- 
nologies that  mitigate  the  observed  problems  such  as  specialized 
interfaces,  and  adaptive  systems.  In  the  report,  we  first  present  an 
overview  of  definitions  and  important  terms  of  supervisory  control 
and  human-agent  teaming.  We  then  discuss  human  performance 
issues  in  supervisory  control  of  multiple  robots  with  regard  to  op- 
erator multitasking  performance,  trust  in  automation,  situation 
awareness,  and  operator  workload.  In  the  following  sections,  we 
review  research  findings  for  specific  areas  of  supervisory  control 
of  multiple  ground  robots,  aerial  robots,  and  heterogeneous  robots 
(using  different  types  of  robots  in  the  same  mission).  In  the  last  sec- 
tion, we  review  innovative  techniques  and  technologies  designed  to 
enhance  operator  performance  and  reduce  potential  performance 
degradations  identified  in  the  literature. 

Index  Terms — Human-agent  teams,  human-robot  interaction 
(HRI),  levels  of  automation  (LOAs),  robotics  control,  supervisory 
control,  unmanned  vehicles,  user-interface  design. 


1.  Introduction 

UNMANNED  vehicles  (UVs),  including  unmanned  air  ve- 
hicles (UAVs)  and  unmanned  ground  vehicles  (UGVs),  are 
becoming  an  essential  part  of  the  battlefield.  The  Armed  Forces 
have  large  programs  for  developing  robotic  systems  [1]  that 
encompass  aerial,  sea,  ground,  and  subterranean  appHcations. 
Future  warfare  will  depend  on  collaboration  among  UVs,  not 
only  within  services  but  also  between  services,  and  eventually, 
among  allied  partners.  Battlefield  collaborations  will  involve 
hundreds  of  UVs,  as  well  as  an  equal  number  of  manned  sys- 
tems requiring  novel  techniques  such  as  call  center  approaches, 
to  monitor  the  unfolding  decision  environment.  The  complex- 
ity and  sheer  number  of  mixed  assets  in  future  operations  will 
require  increased  autonomy  and  problem-solving  capabilities 
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for  unmanned  systems  [2].  Furthermore,  to  maximize  human 
resources,  it  will  be  desirable  to  designate  a  single  operator 
to  supervise  multiple  UVs,  adding  to  his  or  her  already  heavy 
task  loading  [3],  [4].  However,  having  systems  with  automated 
behaviors  introduces  its  own  set  of  problems,  including  overre- 
liance  on  the  automated  system,  potential  situation  awareness 
(SA)  degradations,  and  possible  loss  of  skills  to  perform  the 
tasks  manually  when  automation  fails  [5]-[7]. 

The  purpose  of  this  paper  is  to  review  research  pertaining 
to  the  limitations  and  advantages  of  supervisory  control  for 
UV  systems.  More  importantly,  we  identify  and  discuss  results 
showing  technologies  that  mitigate  the  observed  problems  such 
as  specialized  interfaces  and  adaptive  systems.  In  the  following 
section,  we  first  present  an  overview  of  definitions  and  impor- 
tant terms  of  supervisory  control  and  human-agent  teaming.  We 
then  discuss  human  performance  issues  in  supervisory  control 
of  UVs  with  regard  to  operator  multitasking  performance,  trust 
in  automation,  SA,  and  operator  workload.  In  Sections  III-V, 
we  review  research  findings  for  specific  areas  of  supervisory 
control  of  UAVs,  UGVs,  and  heterogeneous  UVs  (using  both 
UAVs  and  UGVs  in  the  same  mission).  Finally,  in  Section  VI,  we 
review  innovative  techniques  and  technologies  designed  to  en- 
hance operator  performance  and  reduce  potential  performance 
degradations  identified  in  the  literature. 


A.  Supervisory  Control — Overview 

Supervisory  control  of  technology  may  be  defined  from  the 
perspectives  of  human  information  processing  and  the  operator's 
role  in  a  given  task,  by  the  level  of  automation  (LOA)  employed, 
or  by  the  types  of  operator  interactions  with  the  automated 
technology. 

1}  Operator's  Role:  Humans  play  a  variety  of  roles  in  su- 
pervisory tasks  including:  planning,  teaching,  monitoring,  in- 
tervening, and  learning  [8].  These  roles  typically  occur  in  the 
temporal  order  described,  and  may  repeat  throughout  a  super- 
visory task.  Sheridan  [8]  describes  these  roles  in  sequence  as 
follows. 

1)  Planning  the  course  of  action  before  the  automation  is 
activated. 

2)  Instructing  the  computerized  technology  to  perform  a  task 
in  a  particular  manner. 

3)  Monitoring  the  instructed  automation  to  be  sure  it  goes  as 
planned. 

4)  Intervening,  when  necessary,  to  adjust  or  correct  the 
automation. 
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TABLE  I 

Ten  Levels  of  Automation  (LOA)* 


High  10  Full  Autonomy:  The  computer  makes  all  decisions, 
 acts  autonomously,  and  ignores  the  human  operator. 

9  The  computer  informs  the  human  operator,  only  if  it 
 ''decides"  to.  

8  The  computer  informs  the  human  operator,  only  if 
 asked  to.  

7  The  computer  execute  automatically,  then  informs 
 the  human  operator  as  necessary.  

6  Allows  the  human  a  limited  amount  of  time  to  veto 
 an  action  before  it  is  automatically  executed.  

5        The  computer  executes  the  suggestion  with  approval 

 from  the  human  operator.  

 4  The  computer  suggests  one  alternative.  

3  The  computer  narrows  the  decision-making  to  a  few 
 selections.  

2       The  computer  offers  a  complete  set  of  decision  / 

 action  alternatives.  

Low  1  Manual  Operation:  The  computer  offers  no 
assistance  to  the  human  operator,  who  must  make  all 

 decisions  and  take  all  actions.  

^Modified  from  [7,  Table  I]. 


5)  Learning  from  the  performance  and  outcomes  of  the 
automation  in  order  to  improve  planning  for  future 
interactions. 

As  task  complexity  increases,  there  is  greater  need  for  plan- 
ning and  teaching.  In  addition,  the  need  for  monitoring  and 
intervening  depend  on  the  quality  of  planning  and  instruction. 

2)  Level  of  Autonomy:  When  implementing  a  supervisory 
control  task,  the  amount  and  types  of  human  interaction  with  the 
automated  technology  must  be  considered  in  order  to  determine 
the  appropriate  LOA  to  employ.  Parasuraman  et  al.  have  defined 
human  interaction  with  automated  technology  in  terms  of  ten 
"levels  of  automation  of  decision  and  action  selection,"  that  are 
based  on  four  stages  of  human  information  processing. 

1)  Information  acquisition. 

2)  Information  analysis. 

3)  Decision  and  action  selection. 

4)  Action  implementation  [7,  p.  287]. 

The  ten  LOAs  are  defined  in  Table  I,  which  has  been  modi- 
fied from  "Table  I"  appearing  in  [7].  There  are  risks,  benefits, 
and  consequences  associated  with  each  LOA  in  terms  of  the 
associated  mental  workload,  reliance  on  the  automation  (which 
introduces  the  issues  of  trust  and  reliability  of  the  automation), 
and  the  human  operator's  level  of  SA  [7].  Therefore,  it  is  im- 
portant to  plan  for  an  LOA  that  provides  a  balance  of  human 
workload  that  is  challenging  to  the  operator,  yet  manageable, 
and  that  also  provides  a  level  of  SA  sufficient  to  meet  task  per- 
formance goals.  For  example,  automation  level  10  excludes  the 
human  operator  from  making  decisions  and  taking  actions;  thus, 
the  supervisory  monitoring  role  of  the  human  becomes  a  vigi- 
lance task,  which  can  lead  to  operator  complacency,  resulting  in 
human  performance  errors. 

3)  Human-Automation  Interactions:  Supervisory  control 
may  also  be  defined  in  terms  of  the  coordinated  interactions 
that  occur  between  the  human  and  the  automation,  referred  to 
here  as  human-agent  (H-A)  teaming.  Researchers  have  referred 
to  H-A  teaming  as  team  play  and  argue  that  the  interactions 
between  the  operator  and  the  automation  should  be  the  focus 


of  supervisory  task  designs  [5],  [9],  [10].  Researchers  have  em- 
pirically investigated  how  constructs  associated  with  team  play 
(e.g.,  common  ground)  may  be  facilitated  by  the  characteristics 
of  the  automation  during  an  exploration  task.  Stubbs  etal.  [11], 
for  example,  studied  how  common  ground  could  be  facilitated 
among  a  (simulated)  globally  distributed  team  of  operators  and 
an  exploration  robot.  More  specifically,  Stubbs  et  al.  investi- 
gated how  introducing  a  robot  proxy  would  influence  the  degree 
of  collaboration  the  operators  perceived  they  were  engaged  in 
during  the  task.  The  robot  proxy's  role  in  the  task  was  to  assist 
the  operators  with  planning.  Results  indicated  that  those  who 
were  assisted  by  the  robot  proxy  reported  higher  perceptions 
of  collaboration  than  those  who  did  not  use  the  robot  proxy. 
Furthermore,  those  who  used  the  robot  proxy  had  improved 
performance,  more  accurate  mental  models  of  the  robot's  capa- 
bilities, and  were  more  efficient  at  the  exploration  task. 

Concepts  such  as  common  ground,  and  other  human-team 
processes  (e.g.,  coordination  and  communication)  may  provide 
a  comprehensive  understanding  of  H-A  coordination  [11],  [12]. 
H-A  coordination  is  similar  to  human-team  coordination  in 
that  they  both  address  interdependency  and  dynamic  interplay 
among  team  members.  Given  these  similarities,  the  processes 
that  occur  during  human-team  coordination  may  very  likely 
emerge  during  H-A  coordination  [13].  For  example,  communi- 
cation, a  team  process,  emerges  from  the  exchange  of  informa- 
tion between  the  operator  (who  may,  for  example,  request  in- 
formation) and  the  automation  (which  may  provide  information 
about  the  state  of  the  system  and  the  operating  environment). 
In  sum,  the  team  behaviors  that  occur  during  a  supervisory  task 
will  illuminate  relevant  team  processes.  Knowing  what  team 
processes  are  necessary  and  understanding  how  they  will  af- 
fect the  outcome  of  H-A  coordination,  can  inform  the  design  of 
supervisory  tasks. 

The  following  section  discusses  in  detail  some  of  the  major 
human  performance  issues  associated  with  supervisory  control 
of  multiple  UVs.  Particularly,  we  discuss  operator  multitasking 
performance,  trust  in  automation,  SA  (issues  associated  with 
tasking  switching  and  error  diagnosis  and  recovery),  and  oper- 
ator workload. 

II.  Operator  Performance  Issues  in  Supervisory 
Control  of  Multiple  Robots 

A.  Operator  Multitasking  Performance 

In  the  future  battlefield,  soldiers  will  very  likely  be  expected 
to  perform  other  tasks  concurrently  while  operating  a  robot  (or 
multiple  robots)  [14].  Researchers  have  studied  how  charac- 
teristics of  the  automation  (e.g.,  reliability  and  autonomy)  and 
the  task  (e.g.,  complexity)  may  affect  performance  on  these 
concurrent  tasks,  and  the  supervisory  task  as  a  whole.  Manzey 
etal.  [15],  for  example,  investigated  performance  consequences 
of  automated  aids  (for  fault  identification  and  management)  that 
are  occasionally  unreliable  in  a  simulated  multitasking  supervi- 
sory control  task.  The  results  showed  that  automation  benefits 
both  primary  (fault  diagnosis)  and  secondary  task  (response  to 
communications)  performance.  However,  a  significant  automa- 
tion bias  effect  was  observed.  About  half  of  the  participants 
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followed  a  wrong  diagnosis  generated  by  the  automated  sys- 
tem. Additionally,  with  the  highest  LOA,  participants  showed 
degraded  "return-to-manual"  performance.  The  authors,  there- 
fore, recommended  that  medium  LOAs  be  used  if  manual  skills 
need  to  be  maintained. 

Recent  studies  have  investigated  the  consequence  of  task 
complexity  on  multitasking  performance.  These  researchers 
combined  navigation  tasks  concurrently  with  targeting  and  fir- 
ing (search  and  destroy)  tasks  and  intermittent  communication 
tasks  [16],  [17].  Typical  navigation  task  manipulations  included 
perception  (egocentric  versus  exocentric),  attention  (number 
of  robots  to  control  simultaneously),  and  communication  with 
teammates.  To  increase  task  complexity,  search  and  destroy 
tasks  manipulated  attentional  resources  by  increasing  the  num- 
ber of  targets  or  gunner  stations  to  command.  The  multitasking 
study  by  Chen  and  Joyner  [16]  required  an  operator  to  perform 
a  gunner  task,  detecting  and  firing  upon  targets,  while  simulta- 
neously either  managing  a  semiautonomous  UGV  or  teleoper- 
ating  or  monitoring  a  UGV.  A  third  intermittent  task  required 
participants  to  simulate  communication  with  a  gunner  crew.  Re- 
sults showed  that  overloading  the  operator's  mental  capacity 
with  multiple  complex  tasks  simultaneously  led  to  performance 
decrements  (decreased  target  detection);  however  when  (semi- 
)  autonomy  was  given  to  robotic  entities,  the  operator  could 
focus  cognitive  resources  on  complex  tasks  (target  detection 
and  communication/reasoning  tasks)  while  simply  monitoring 
(rather  than  controlling)  the  semiautonomous  robots,  interven- 
ing only  when  necessary. 

The  impact  of  task  complexity  was  also  investigated  by  Cum- 
mings  and  Guerlain  [17]  in  terms  of  the  number  of  entities 
one  can  operate  under  multitasking  conditions.  More  specifi- 
cally, the  researchers  tried  to  determine  the  approximate  num- 
ber of  autonomous  vehicles  (missiles)  that  one  operator  could 
efficiently  command  and  control  simultaneously,  under  a  low- 
or  high-tempo  multitask  condition  (performing  a  missile  retar- 
geting task  and  an  intermittent  communication  task).  Results 
showed  that  16  missiles  were  the  limit  to  control,  beyond  which 
point  performance  (reallocation  of  missiles  to  time- sensitive  tar- 
gets) began  to  degrade.  Cummings  and  Guerlain  cited  that  their 
findings  corresponded  with  air  traffic  control  (ATC)  studies, 
where  17  managed  aircraft  was  the  limit  that  operators  could 
handle  well.  Other  studies  have  concluded  that  a  much  fewer 
number  of  UAVs  could  be  controlled  simultaneously  [18];  how- 
ever, it  is  important  to  consider  the  LOAs  applied  to  the  au- 
tonomous entities.  In  many  cases,  the  more  manual  operation 
that  was  demanded,  the  fewer  entities  that  could  effectively  be 
supervised  [19]. 

These  studies  show  that  both  the  characteristics  of  the  au- 
tomation (e.g.,  reliability)  and  the  characteristics  of  the  task 
(e.g.,  complexity)  will  influence  an  operator's  ability  to  multi- 
task during  a  supervisory  control  task.  Other  factors,  specifically 
related  to  the  characteristics  of  the  environment  in  which  the  task 
occurs,  may  impact  supervisory  control  as  a  whole.  One  example 
of  such  factors  is  the  degrees  of  coherence  (comprehensiveness, 
consistency,  and  rational  soundness)  and  correspondence  (ac- 
curacy) of  the  information  available  for  these  tasks  [20].  Since 
coherence  is  a  prerequisite  to  correspondence,  Mosier  et  al.  [20] 


studied  the  effects  of  coherence  (information  congruence)  in  the 
context  of  time  pressure  and  operator  confidence.  In  their  study, 
airplane  pilots  were  found  to  make  quicker,  less-accurate  deci- 
sions under  time  pressure,  as  they  did  not  utilize  all  of  the  cues 
available  to  them.  Instead,  they  tended  to  use  cognitive  heuris- 
tics such  as  anchoring  the  most  salient  cues  to  satisfice,  rather 
than  take  the  time  necessary  to  make  a  better-informed  deci- 
sion. Satisficing  was  also  observed  in  the  supervisory  command 
and  control  study  by  Cummings  and  Guerlain  [17]  in  which 
participants,  who  were  under  a  time  constraint,  had  to  select 
one  of  several  missiles  to  redirect  toward  a  target.  Participants 
often  redirected  a  missile  that  was  sub-optimal,  yet  sufficient, 
to  complete  the  mission. 

B.  Trust  in  Automation 

Trust  in  automation  is  a  ubiquitous  construct,  particularly  with 
respect  to  performance  during  supervisory  control  tasks.  Trust 
in  automation,  however,  may  be  misleading  because  trust  has 
the  connotation  of  a  prescribed  behavior.  Calibration,  instead,  is 
a  more  fitting  term  because  it  suggests  that  during  a  supervisory 
control  task,  operators  intervene  only  when  they  have  reason  to 
believe  their  own  decisions  (od)  are  superior  to  the  automated 
system's  decisions  (ad).  In  decision  theoretic  terms,  operators 
choose  to  use  automated  systems  when  the  probability  of  a 
correct  decision,  /?(c),  meets  the  following  criterion:  Intervene 
if{p-od(c))  >  (p-ad(c))  [21]. 

Operators,  however,  have  a  difficult  time  assessing  their  own 
accuracy.  In  general,  humans  tend  to  be  poorly  calibrated — often 
overestimating  their  own  abilities  [22] .  This  implies  that  humans 
will  overvalue  their  own  decisions  in  comparison  to  automated 
solutions.  However,  the  opposite  tendency  for  humans  to  over 
rely  on  automated  systems  has  been  shown  by  a  number  of 
researchers  [23]-[29]. 

Researchers  have  studied  factors  affecting  the  under  or 
overuse  of  automation  that  encompasses  both  extrinsic  factors, 
in  that  they  are  related  to  the  characteristics  of  the  environment 
or  the  automation,  and  intrinsic  factors  in  that  they  are  related 
to  the  characteristics  of  the  operator. 

1 )  Extrinsic  Factors:  Researchers  suggest  that  the  psycho- 
logical context  of  the  decision  determines  the  tendency  of  the 
operator  to  disuse  (under  rely)  or  misuse  (over  rely  on)  au- 
tomated systems  [6].  In  a  series  of  experiments,  Dzindolet 
et  al.  [30]  showed  that  by  a  simple  change  in  decision  order, 
disuse  of  an  automated  target  recognition  device  changed  to 
misuse.  If  participants  made  a  decision  before  being  informed 
of  the  automated  solution,  they  tended  to  rely  on  their  own 
decisions  even  when  they  were  suboptimal,  whereas,  in  a  re- 
lated experiment  using  a  similar  paradigm,  participants  tended 
to  over  rely  on  the  device  whenever  the  automated  solution  was 
presented  at  the  same  time  as  the  target  scene.  One  explanation 
is  that  participants  were  attempting  to  reduce  their  cognitive 
load  to  maintain  reasonable  timeliness.  However,  such  a  strat- 
egy was  not  useful  in  the  first  experiment  (automated  solutions 
being  shown  after  the  participants  made  their  decisions)  be- 
cause considering  an  alternative  would  increase  workload  by 
requiring  operators  to  reconsider  their  original  decisions  [31]. 
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The  workload  hypothesis  is  supported  by  complacency  research 
that  indicates  operators  misuse  automation  in  a  multitasking 
environment  but  not  in  single  task  environment  [27]. 

The  reliability  of  the  automated  system  also  impacts  oper- 
ator disuse  or  misuse.  A  number  of  researchers  reported  that 
target  cueing  is  ineffective  below  a  certain  level  of  reliabil- 
ity (approximately  60%-70%)  [32],  [33].  In  their  metaanalytic 
study,  Wickens  and  Dixon  found  that  "a  reliability  of  0.70  was 
the  'crossover  point'  below  which  unreliable  automation  was 
worse  than  no  automation  at  all"  [32,  p.  2].  Reliable  automation 
enhances  performance  [34];  however,  even  reliable  aids  tend  to 
be  disused  if  the  few  mistakes  the  aid  makes  are  obvious  such  as 
failing  to  detect  a  target  when  it  is  plainly  visible  [35].  However, 
the  converse,  automation  paradox  is  also  evident;  operators  tend 
to  over  rely  more  on  highly  accurate  aids  when  advisories  are  in- 
correct, i.e.,  operators  are  more  complacent  when  they  develop 
trust  in  automation  with  high  reliability  because  the  possibility 
that  the  aid  will  mislead  them  is  perceived  as  minimal  [34],  [36]. 

The  type  of  unreliability  has  an  important  impact  on  the  op- 
erator's perception  of  and  response  to  system  alerts.  Cueing 
systems  for  the  automated  systems  are  often  false-alarm  prone 
(FAP)  or  miss-prone  (MP)  based  on  the  threshold  settings  of 
the  alert.  Wickens  et  al.  [37]  showed  that  the  operator's  auto- 
mated task  (system-failure  monitoring)  performance  degraded 
when  the  false  alarm  (FA)  rate  of  the  alerts  for  the  automated 
task  was  high.  In  other  words,  high  FA  rate  reduced  operator's 
compliance  with  automation  (compliance  was  defined  as  "the 
tendency  to  agree  with  an  automated  aid  when  it  provides  an 
alert"  [38,  p.  1941].  Conversely,  when  the  miss  rate  was  high, 
performance  on  a  concurrent  task  was  affected  more  than  the 
automated  task,  because  the  operator  had  to  allocate  more  vi- 
sual attention  to  monitor  the  automated  task.  In  other  words, 
high  miss  rate  reduced  operator  reliance  on  automation  (re- 
liance was  defined  as  "the  operator's  assumption  that  a  system 
is  functioning  normally  while  the  alert  is  silent"  [38,  p.  1941]. 
Similarly,  Dixon  and  Wickens  [39]  showed  that  FAs  and  misses 
affected  compliance  and  reliance,  respectively,  and  their  effects 
appeared  to  be  relatively  independent  of  each  other.  In  contrast, 
Dixon  et  al.  [40]  showed  that  FAP  automation  hurt  "perfor- 
mance more  on  the  automated  task  than  did  MP  automation 
(e.g.,  the  "cry  wolf  effect)  and  hurt  performance  (both  speed 
and  accuracy)  at  least  as  much  as  MP  automation  on  the  concur- 
rent task"  (pp.  570-571).  FAP  automation  was  found  to  affect 
both  operator  compliance  and  reliance,  while  MP  automation 
affected  only  operator  reliance.  The  authors  suggested  that  the 
FAP  automation  had  a  negative  impact  on  reliance  because  of  op- 
erator's overall  reduced  trust  in  the  automated  system.  Similarly, 
Wickens  et  al.  [41]  demonstrated  a  greater  cost  associated  with 
FAP  automation  (than  with  MP  automation),  which  affected 
both  the  automated  and  concurrent  tasks. 

Lees  and  Lee  [42]  showed  that  "unnecessary  alarms"  (i.e., 
alerts  that  are  legitimate  but,  due  to  the  peculiarities  of  the  situ- 
ation, do  not  require  operator  compliance)  foster  the  operators' 
trust  in  the  automated  system  and  actually  enhance  their  compli- 
ance with  the  alerts  rather  than  reducing  it.  Lees  and  Lee's  data 
suggest  that  the  three  dimensions  of  trust  (utility,  predictability, 
and  intent)  need  to  be  considered  beside  the  traditional  descrip- 


tion of  alarms  according  to  signal  detection  theory  (i.e.,  FAP 
versus  MP). 

Another  distinction  important  to  disuse  or  misuse  of  automa- 
tion is  the  difference  between  evaluation  errors  and  intent  errors. 
For  evaluation  errors,  the  operator  misperceives  the  optimal  so- 
lution and  commits  a  calibration  error.  For  intent  errors,  the 
operator  is  aware  of  the  aids  superiority  but  still  chooses  to 
"disuse"  automation  in  order  to  maintain  control  over  the  de- 
cision environment.  For  example,  intent  errors  occurred  when 
participants  were  aware  of  the  superiority  of  an  automated  tar- 
get detection  aid  but  84%  of  them  still  tended  to  disuse  the  aid 
leading  to  suboptimal  targeting  decisions  [43] . 

Many  of  the  above  results  suggest  that  trust  (or  calibration) 
is  an  extrinsic  factor  determined  by  operators'  perception  of 
the  task  environment.  However,  intent  errors  and  FAs,  indicate 
"trust"  decisions  are  also  based  on  the  operators'  belief  in  the 
importance  of  personally  controlling  decision  outcomes  [24], 
[43],  which  can  be  labeled  as  intrinsic  factors. 

2)  Intrinsic  Factors:  Chen  and  Terrence  [24]  showed  that 
there  is  a  strong  interaction  between  the  type  of  automation 
unreliability  and  participants'  self-assessed  attentional  control. 
Overall,  it  appears  that  for  high  attentional  control  participants, 
FAP  alerts  were  more  detrimental  than  MP  alerts,  due  to  disuse 
of  automation.  For  low  attentional  control  participants,  con- 
versely, MP  automation  was  more  harmful  than  FAP  automa- 
tion, due  to  misuse  of  (i.e.,  overreliance  on)  automation.  Their 
results  are  consistent  with  past  research  that  self-confidence 
is  a  critical  factor  in  mediating  the  effect  of  trust  (in  automa- 
tion) on  reliance  (on  the  automatic  system)  [44],  [45].  Lee  and 
Moray  found  that  when  self-confidence  exceeded  trust,  opera- 
tors tended  to  use  manual  control.  When  trust  exceeded  self- 
confidence,  automation  was  used  more. 

Lee  and  See  [45]  characterize  the  interactions  within  a  super- 
visory control  task  in  terms  of  how  human  teams  interact.  They, 
therefore,  define  trust  as  the  attitude  that  an  agent  will  help 
achieve  an  individual's  goal  in  a  situation  characterized  by  un- 
certainty and  vulnerability.  When  humans  perceive  automated 
systems  as  team  members,  their  attitudes  of  misuse  or  disuse 
should  develop  as  operators  become  familiar  with  the  system; 
therefore,  implying  that  the  "trust"  decision  is  dependent  on  the 
attitude  of  humans  towards  automated  systems.  This  intrinsic 
definition  has  interesting  implications  suggesting  that  humans 
view  automated  systems  anthropomorphically.  This  begs  the 
question  whether  we  can  design  systems  not  only  to  perform 
human-like  tasks,  but  to  actually  act  like  humans.  For  example, 
if  robots  can  be  made  to  be  perceived  as  human,  then  teaming 
relations  would  not  only  be  more  natural  but  also  instinctive  and 
synergistic  like  a  well-honed  basketball  team.  Research  to  date 
is  sparse  but  suggestive.  Based  on  their  research,  Arkin  [46] 
and  his  colleagues  posit  that,  in  the  future,  robots  can  be  made 
not  only  to  react  to  human  cues  but  do  so  in  a  predictable 
and  human-like  manner — even  to  extent  of  developing  ethical 
behaviors.  This  would  allow  humans  and  robots  to  share  men- 
tal models  of  not  only  the  external  environment  but  also  share 
each  others'  models  [47].  However,  the  research  in  this  area 
is  not  mature;  humanoid  robots  or  even  robots  that  act  like 
dogs  can  operate  successfully  only  in  constrained  laboratory  or 
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commercial  environments.  Currently,  humans  control  or  moni- 
tor unmanned  systems  as  sophisticated  equipment  rather  than  as 
team  members,  showing  more  trust  in  a  perceived  human  peer 
than  a  machine  peer  with  the  same  capabilities  [21]. 

Studying  both  the  extrinsic  and  intrinsic  factors  affecting  the 
use  or  disuse  of  automation  is  essential  for  developing  training 
tools  for  reducing  errors  associated  with  disuse  or  misuse.  Beck 
et  ah  [43]  indicate  that  a  combination  of  performance  feedback 
and  scenario  training  can  reduce  both  appraisal  and  intent  errors 
suggesting  that  different  training  regimes  may  be  necessary  for 
extrinsic  (feedback)  and  intrinsic  (scenario  training)  sources  of 
automation  usage  errors. 

Recently,  researchers  have  suggested  that  the  trust  in  automa- 
tion research  addressed  above  is  a  foundation  for  modeling  trust 
in  human-robot  interactions  (HRIs);  however,  a  new  model  of 
trust  must  be  developed  [48].  Desai  et  al.  suggested  that  the 
model  account  for  the  differences  between  current  systems,  and 
those  used  in  prior  research;  for  example,  the  newer  systems' 
lack  of  reliability,  and  the  more  dynamic  nature  of  the  systems 
in  terms  of  adjustable  autonomy  and  their  ability  to  carry  out 
multiple  tasks. 

C.  Situation  Awareness 

One  of  the  most  critical  factors  for  achieving  effective  su- 
pervisory control  of  multiple  UVs  is  maintaining  adequate  SA 
of  the  overall  tasking  environment  as  well  as  individual  UVs. 
There  are  three  levels  of  SA  as  defined  by  Endsley  [49]:  1) 
perception  of  data  and  environmental  elements;  2)  comprehen- 
sion of  the  current  situation;  and  3)  projection  of  future  states 
and  events.  Changes  in  the  environment  that  may  affect  the 
plans  for  the  robots  need  to  be  detected  and  the  plans  need  to 
be  modified  in  a  timely  fashion.  While  updates  of  information 
are  needed,  recent  studies  have  also  shown  that  interrupting  a 
primary  task  (i.e.,  supervisory  control  of  UVs)  with  an  intermit- 
tent task  (e.g.,  communication  messages)  can  have  a  negative 
impact  on  SA  [50],  [51].  For  example,  Cummings  [50]  found 
that  instant  messages  diverted  participants'  attention  from  their 
primary  task  (simulated  supervisory  control  of  Tomahawk  mis- 
siles), thus  reducing  their  SA  when  returning  their  focus  to  the 
primary  task.  Therefore,  the  challenge  is  to  identify  the  means 
by  which  operators  can  most  effectively  maintain  SA. 

S  A  may  also  be  affected  perceptually  as  a  result  of  the  change 
blindness  phenomenon,  which  is  the  inability  to  perceptually  at- 
tend to  a  change  in  one's  environment.  Parasuraman  et  al.  [52] 
examined  change  blindness  in  the  context  of  a  supervisory  con- 
trol task,  in  which  participants  were  asked  to  monitor  a  UAV 
and  a  UGV  video  feed  in  a  reconnaissance  tasking  environment. 
Participants  performed  four  tasks  in  this  experiment,  including 
target  detection  and  route  planning  (primary  tasks),  a  commu- 
nications task  to  evaluate  SA,  and  a  change  detection  task.  The 
routes  for  the  UAV  and  the  UGV  were  preplanned,  and  the  only 
time  the  participant  controlled  the  UGV  was  when  it  was  nec- 
essary to  navigate  around  an  obstacle  in  the  environment.  The 
primary  tasks  were  interrupted  by  both  the  verbal  communi- 
cation task  and  the  change  detection  task.  The  latter  required 
participants  to  indicate  each  time  they  noticed  that  an  icon  of  a 


target  they  had  previously  detected  had  unexpectedly  changed 
position  on  a  map  grid.  Half  of  these  changes  occurred  dur- 
ing a  "transient  event,"  when  the  UGV  stopped  and  its  status 
bar  flashed,  while  the  remaining  four  changes  occurred  while 
participants  were  focused  on  the  UAV  monitoring  task.  Parasur- 
aman et  al  [52]  found  that  participants'  accuracy  at  detecting 
changes  related  to  the  position  of  the  target  icons  was  very  low, 
especially  during  the  transient  events.  Their  results  suggest  that 
change  blindness  occurred  most  frequently  when  a  distracter 
(a  transient  event)  was  present,  but  also  occurred  while  partic- 
ipants shifted  their  attention  from  the  UAV  monitoring  task  to 
the  UGV  monitoring  task.  These  results  also  suggest  that  task 
switching  during  a  robot  supervisory  task  may  incur  change 
blindness,  which  by  its  very  nature  affects  an  operator's  SA  (ad- 
ditional information  on  task  switching  is  provided  in  the  next 
paragraph).  According  to  Norman's  [53]  seven  stages  of  user 
activity,  interruptions  incur  the  greatest  cognitive  costs  during 
the  planning  phases  (intention  forming  and  action  planning)  as 
well  as  the  evaluation  phases  (outcome  interpretation  and  as- 
sessment). Thus,  interface  designers  should  account  for  this,  so 
that  primary  tasks  are  only  interrupted  during  emergency  situa- 
tions or  during  moments  of  low  workload  (e.g.,  after  evaluation 
is  completed  or  before  initiating  a  new  plan).  However,  alerts 
should  be  provided  to  the  operator  indicating  the  changes  to  the 
interface  and  the  degree  of  importance  of  the  changes  [54]. 

1)  Task  Switching:  Simultaneous  control  of  multiple  UVs 
may  require  the  operator  to  switch  attention/control  among  the 
vehicles  from  time  to  time.  Basic  research  on  costs  of  task 
switching  consistently  shows  that  people's  responses  tend  to 
be  substantially  slower  and  more  error-prone  after  task  switch- 
ing [55],  [56].  There  is  some  evidence  that  this  cost  may  be 
reduced  if  the  participants  have  a  chance  to  prepare  for  the 
switch  or  receive  task  switching  cues  [55],  [56].  In  the  context 
of  HRI,  research  has  been  conducted  to  investigate  the  effects 
of  task  switching  on  SA  and  operator  performance  [57]-[59]. 
Squire  et  al.  [58]  studied  operators'  response  time  as  it  related 
to  interface  type  (options  available  to  the  user)  and  task  switch- 
ing, as  well  as  strategy  switching  (offensive  versus  defensive, 
in  the  context  of  a  RoboFlag- simulated  game).  Task  switching 
was  shown  to  slow  response  time  by  several  seconds,  espe- 
cially when  automation  was  involved.  Likewise,  Squire  et  al. 
found  that  response  time  increased  by  several  seconds  when 
participants  switched  between  offensive  and  defensive  strate- 
gies. When  participants  were  provided  an  interface  that  allowed 
the  flexibility  to  choose  between  a  fixed  sequence  of  automated 
actions  or  selectable  waypoint-to-waypoint  movement,  mission 
time  was  reduced  in  spite  of  task  or  strategy  switching.  Crandall 
et  al.  [57]  examined  the  amount  of  interaction  time  (IT)  versus 
non-IT,  or  "neglect  tolerance,"  to  develop  a  predictive  model 
to  determine  the  number  of  robots  (heterogeneous  or  homoge- 
neous) that  may  be  effectively  monitored  simultaneously  by  a 
single  operator,  given  the  requirements  of  a  particular  interactive 
task.  Crandall  et  al.  suggested  that  a  predictive  analysis  such  as 
theirs  might  be  useful  in  the  context  of  task  switching. 

2)  Error  Diagnosis  and  Recovery:  Frequently,  changes  in 
the  environment  may  require  the  operator  to  modify  his/her 
plans  for  the  UVs.  Muthard  and  Wickens  [60]  evaluated  the 
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effects  of  automation  on  pilot's  performance  in  plan  monitoring 
and  revision.  They  found  that  pilots  only  detected  about  30%  of 
the  experimenter-induced  changes,  which  should  have  resulted 
in  flight  plan  revisions.  Mumaw  et  al.  [61]  showed  an  even  more 
alarming  inadequacy  in  monitoring  performance.  In  their  study, 
pilots  only  detected  about  3%  (1  of  32  total  cases)  of  unex- 
pected changes  to  the  mode  of  an  automation  aid.  Indeed,  it  has 
been  well  documented  that  operators  are  frequently  unaware  of 
mode  changes  when  interacting  with  automation  systems  and, 
therefore,  are  confused  about  the  systems'  behaviors  [62].  In 
fact,  data  show  that  even  if  changes  in  the  environment  are  de- 
tected, operators  may  have  difficulty  interpreting  the  relevance 
of  the  changes  and  their  effects  on  the  existing  plans  [49].  Ac- 
cording to  the  National  Transportation  Safety  Board  (1994,  as 
cited  in  [60]),  nearly  two-thirds  of  aviation  accidents  caused  by 
faulty  human  decision  making  can  be  attributed  to  pilots'  failure 
to  revise  their  flight  plans.  In  Muthard  and  Wickens  [60],  pilots 
failed  to  modify  their  plans  based  on  environmental  changes 
on  nearly  one-third  of  the  trials.  Plan  continuation  errors  were 
especially  pronounced  when  there  was  an  unreliable  automation 
aid  as  compared  with  no  aid  present. 

D.  Operator  Workload 

An  operator's  perceived  workload  can  impact  his/her  per- 
formance in  an  UV  supervisory  task  relative  to  the  LOA  as- 
signed to  the  robotic  entities  being  supervised.  The  LOA  of  a 
robotic  entity  further  influences  operator  compliance  and  re- 
liance on  the  automated  robots.  Ruff  et  al.  [19]  examined  per- 
ceived workload  and  found  that  when  given  complex  tasks, 
operators  tended  to  prefer  autonomy  that  provided  an  LOA  ex- 
hibiting "management-by-exception"  (approximately  level  6  in 
Table  I).  Although  this  LOA  produced  performance  decrements, 
it  lowered  perceived  workload  as  participants  demonstrated  sat- 
isficing.  In  contrast  to  "management-by-exception,"  Ruff  et  al. 
found  that  perceived  workload  increased  when  participants  were 
subjected  to  automation  that  was  less  than  perfectly  accurate. 
Their  lack  of  trust  in  the  automation  led  participants  to  take  extra 
time  to  double-check  the  system's  accuracy  in  a  "management- 
by-consent"  condition  (approximately  LOA  3, 4,  or  5  in  Table  I). 
There  are  also  individual  differences  associated  with  perceived 
workload.  For  example,  Mogford  et  al.  [63]  found  that  individ- 
ual operator  differences  were  a  mediating  factor  of  perceived 
mental  workload,  along  with  equipment  quality  and  ATC  oper- 
ator strategies,  in  a  model  of  ATC.  More  on  operator  workload 
will  be  discussed  in  Sections  III  and  IV. 

In  summary,  the  findings  in  this  section  suggest  that  human 
operator's  supervisory  control  task  performance  can  be  affected 
by  factors  such  as  the  reliability  of  the  automated  systems,  in- 
dividual differences  (e.g.,  operator's  attentional  control  skills 
and  confidence  in  his/her  own  manual  control  abilities),  infor- 
mational display  (e.g.,  timings  of  information  being  presented 
and  how  the  information  is  displayed),  and  concurrent  workload 
(e.g.,  task  switching,  replanning,  and  additional  tasks  such  as 
communication).  The  following  sections  review  research  find- 
ings in  specific  areas  of  supervisory  control  of  UAVs,  UGVs, 
and  heterogeneous  UVs  (using  different  types  of  UVs  in  the 


same  mission),  followed  by  a  review  of  user-interface  designs 
for  supervisory  control  of  multiple  robots. 

III.  Supervisory  Control  of  Unmanned  Aerial  Vehicles 

Two  often  used  methods  of  supervisory  control  of  UAVs  are 
management  by  consent  (MBC)  and  management  by  exception 
(MBE),  which  have  been  researched  extensively  in  the  manned 
aviation  environment  [64].  MBC  requires  the  automation  to  ask 
for  explicit  consent  from  the  human  operator  before  taking  any 
actions;  MBE,  on  the  other  hand,  allows  the  automation  to  ini- 
tiate/perform actions  unless  overruled  by  the  human  operator. 
In  their  survey  of  airline  pilots,  Olson  and  Sarter  [65]  found 
that  MBC  was  preferred  by  most  pilots  under  normal  circum- 
stances, as  it  gave  them  more  control  over  the  automated  sys- 
tems. However,  MBE  was  preferred  when  the  pilots  were  under 
time  pressure  and  higher  workload,  or  when  the  task  was  highly 
complex  or  of  low  criticality.  Olson  and  Sarter  [64]  conducted 
a  simulation  experiment  to  further  examine  pilot  performance 
when  interacting  with  MBC  systems.  The  results  showed  that, 
under  time  constraints,  pilots  often  had  difficulties  in  detect- 
ing conflicts  before  giving  consent  to  the  automation,  especially 
when  the  automation  did  more  than  expected  by  the  pilots. 

Comparisons  of  the  effectiveness  of  MBC  and  MBE  were  also 
conducted  in  the  unmanned  aviation  environment  [19],  [66], 
[67].  Ruff  et  al.  [19]  evaluated  the  effects  of  LOA,  decision-aid 
reliability,  and  number  of  UAVs  under  control  (1,  2,  or  4)  on 
the  human  operator's  target-acquisition  task  performance.  They 
found  that,  overall,  an  MBC-type  decision  aid  provided  the  high- 
est mission  efficiency  (total  number  of  targets  destroyed  divided 
by  the  total  number  of  missiles  fired)  and  resulted  in  the  best  sur- 
vivability performance  (number  of  UAV  hit  points  sustained). 
However,  the  authors  reported  that  automation  level  interacted 
with  number  of  UAVs  controlled  and  reliability  of  decision  aid 
for  subjective  measures  of  SA,  perceived  workload,  and  trust  in 
the  automation  system.  They  found  that  SA  was  highest  in  the 
MBC  condition,  followed  by  manual  operation,  and  finally,  the 
MBE  condition.  As  the  number  of  controlled  UAVs  increased, 
SA  degraded  for  all  three  LOA  conditions,  with  the  manual 
condition  having  the  most  severe  degradation.  Furthermore,  the 
higher  the  LOA,  the  more  benefit  of  vigilance  (e.g.,  system  mon- 
itoring) and  workload  relief  it  provided  during  complex  tasks 
(when  the  number  of  UAVs  increased  and/or  when  reliability 
decreased).  For  example,  in  the  MBC  condition  at  95%  reliabil- 
ity, participants  experienced  higher  workload  (compared  with 
the  MBE  condition)  as  a  result  of  double-checking  to  be  sure 
the  automation  was  accurate.  In  the  context  of  trust  in  the  sys- 
tem, researchers  found  that  it  increased  in  both  the  MBC  and 
the  manual  conditions  as  the  number  of  UAVs  increased  when 
reliability  was  perfect,  with  the  MBC  providing  the  most  trust. 
However,  when  the  reliability  was  not  perfect  (i.e.,  95%  reli- 
able), both  MBC  and  MBE  resulted  in  lower  trust  ratings  as  the 
number  of  UAVs  increased. 

Ruff  et  al.  [66]  conducted  another  study  to  compare  the  ef- 
fectiveness of  MBC  and  MBE  systems  in  a  UAV  control  setting 
(2  or  4  UAVs).  In  this  study,  the  MBC  system  proposed  route 
replans  and  target  identifications.  The  MBE  system,  on  the  other 
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hand,  automatically  implemented  proposed  actions  after  a  pre- 
determined time  period,  unless  the  operator  overruled.  Overall, 
regardless  of  experimental  condition,  participants  did  not  utilize 
the  automation  aiding  system  much  at  all.  When  under  greater 
time  constraints,  they  utilized  it  more  often,  but  still  rarely.  As 
contrary  to  the  previous  study,  MBC  did  not  result  in  signif- 
icantly better  performance  than  MBE.  However,  with  MBE, 
participants'  perceived  workload  increased  as  time  pressure  in- 
creased. With  MBC,  in  contrast,  perceived  workload  remained 
at  a  similar  level  with  an  increased  time  constraint.  In  other 
words,  although  MBE  was  hypothesized  to  reduce  workload 
under  high  time  pressure,  the  opposite  was  found. 

Cummings  and  Mitchell  [67]  investigated  LOA  in  the  context 
of  controlling  four  UAVs  for  target  acquisition  tasks.  The  LOAs 
employed  in  the  study  roughly  corresponded  to  levels  1,  2,  4, 
and  6  in  Parasuraman  et  al.  [7].  Level  of  replanning  required  in 
reaction  to  unexpected  events  was  also  manipulated  (high  versus 
low).  They  found  that  under  the  high  replanning  condition,  op- 
erators performed  the  worst  with  the  "active"  level  automation 
(corresponding  most  closely  to  MBC).  The  authors  described 
this  performance  decrement  as  "cognitive  saturation"  and  ob- 
served that  operators  were  unable  to  correctly  assimilate  data 
from  various  sources,  weigh  uncertainty  in  the  solution  space, 
and  prioritize  their  tasks.  Operators  had  the  highest  level  of 
level- 1  SA  (perception)  and  level-2  SA  (comprehension)  with 
the  "superactive"  level  automation  (MBE)  but  not  the  level-3 
SA  (future  projection).  In  fact,  operators  exhibited  automation 
complacency  and  erroneously  destroyed  more  targets  than  they 
did  when  using  automation  with  lower  LOAs.  This  finding  of 
complacency  is  consistent  with  what  has  been  reported  in  the 
literature  and  is  an  important  issue  that  user-interface  designers 
of  automation  systems  need  to  consider  [16],  [27]-[29]. 

Levinthal  and  Wickens  [38]  also  investigated  the  effects  of 
imperfect  automation  on  robotics  operator's  target  detection 
performance  when  controlling  multiple  (2  or  4)  UAVs.  This 
study  compared  Meyer's  [68]  independence  hypothesis  (which 
posited  that  FAP  and  MP  automations  have  qualitatively  differ- 
ent effects  on  operator  dependence  on  the  automated  systems) 
to  the  model  that  FAP  automation  hurts  more  than  MP  automa- 
tion. Their  results  were  in  favor  of  the  independence  hypothesis 
as  participants  in  a  MP  condition  (60%  reliability  with  3:1  like- 
lihood of  misses  over  FAs)  showed  an  increase  in  compliance 
and  a  decrease  in  reliance  whereas  participants  in  a  FAP  con- 
dition (60%  reliability  with  3:1  likelihood  of  FAs  over  misses) 
showed  a  decrease  in  compliance  and  an  increase  in  reliance. 
However,  it  was  found  that  FAP  aids  resulted  in  delayed  re- 
sponse times  compared  to  the  MP  aids  and  90%  reliability  aids. 
The  authors,  therefore,  concluded  that  FAP  aids,  overall,  were 
more  disruptive  to  operator  performance  than  MP  aids. 

Some  researchers  examined  operator  performance  and  work- 
load in  control  of  large  numbers  of  UAVs  [69].  For  example, 
Miller  [18]  modeled  the  operator  workload  whose  task  is  to 
authorize  weapon  release  for  targets  identified  by  UAVs.  His 
model  shows  that,  under  anticipated  target  densities,  the  opera- 
tor would  become  overloaded  when  controlling  13  UAVs,  even 
if  the  weapon  release  authorization  is  the  only  task  the  operator 
has  to  perform.  In  an  empirical  study,  Galster  et  al.  [69]  exam- 


ined operator  performance  in  a  supervisory  control  task  of  4,  6, 
or  8  UAVs.  Overall,  participants  performed  well  on  their  pri- 
mary task  (selecting  the  correct  targets  and  the  highest  priority 
targets  to  process).  Results  suggested  that  although  the  increase 
in  number  of  UAVs  slightly  impacted  performance  (negatively), 
the  number  of  targets  (not  the  number  of  UAVs)  had  a  higher 
impact  on  perceived  workload.  Secondary  (monitoring)  task 
performance,  on  the  other  hand,  was  significantly  worse  for  the 
8  UAV  condition  than  for  the  4  and  6  UAV  conditions. 

Overall,  the  findings  reviewed  in  this  section  suggest  that  the 
benefits  of  MBE  versus  MBC  automation  for  supervisory  con- 
trol of  multiple  UAVs  are  not  always  consistent  and  they  are 
affected  by  factors  such  as  the  difficulty  level  of  the  task  (e.g., 
target  density)  and  the  type  of  automation  unreliability  (e.g., 
FAP  or  MP).  Additionally,  operators  may  exhibit  complacency 
under  high  task  load  when  using  systems  with  higher  LOAs. 
Studies  reviewed  in  the  "Trust  in  Automation"  segment  of  the 
previous  section  suggest  that  factors  such  as  individual  differ- 
ences also  play  an  important  role  in  how  operators  interact  with 
highly  automated  systems. 

IV.  Supervisory  Control  of  UGVs 

Supervisory  control  of  multiple  UGVs  has  been  examined  in 
contexts  of  search  and  rescue  tasks  [70],  [71]  and  other  explo- 
ration and  reconnaissance  tasks  [72],  [73].  Olsen  and  Wood  [74] 
proposed  an  equation  to  calculate  fan-out  (FO;  the  number  of 
robots  a  human  operator  can  control  simultaneously  [span  of 
control])  based  on  activity  time  (AT;  the  time  that  a  robot  is 
active)  and  IT  (the  time  that  it  takes  for  a  human  operator  to  in- 
teract with  a  robot).  The  authors  also  conducted  four  simulated 
experiments  to  test  the  equation.  Results  showed  that,  when  18 
robots  were  available  for  a  target  search  task,  FO  for  the  type 
of  robots  with  the  highest  autonomy  was  as  high  as  9  for  the 
less-cluttered  tasking  environment  (compared  to  5  in  the  more- 
cluttered  environment).  Olsen  and  Wood  concluded  that  their 
FO  equation  could  accurately  model  operator  span  of  control 
in  many  cases;  however,  FO  might  change  due  to  other  task 
characteristics  and  complexities  in  the  environment. 

Riley  and  Strater  [75]  investigated  navigating  two  UGVs 
through  a  maze  while  manipulating  the  control  mode  (serial, 
parallel,  and  two  parallel  conditions  involving  manual  control 
of  one  robot  and  a  varying  LOA  in  the  second,  supervised  robot). 
Performance  measures  included:  SA  (measure  of  awareness  of 
current  and  future  status  of  robotics  task  using  the  Situation 
Awareness  Global  Assessment  Technique  [SAGAT]),  workload 
(NASA-TLX),  and  performance  navigating  the  robots  (time-to- 
task  completion)  as  well  as  monitoring  performance  (detection 
of  diagnostic  failures).  Monitoring  performance  was  positively 
correlated  with  SA  scores,  and  participants  in  the  serial  control 
condition  exhibited  slightly  higher  SA  scores  than  the  paral- 
lel conditions.  However,  the  parallel  control  condition  resulted 
in  the  best  navigation  performance  and  the  lowest  perceived 
workload. 

Participants  in  Chadwick's  [76]  study  were  responsible  for 
controlling  1,  2,  or  4  semiautonomous  UGVs  simultaneously. 
Operators  were  assessed  on  monitoring,  responding  to  cued 
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decision  requests,  and  detecting  contextual  errors,  the  latter  of 
which  occurred  when  the  robot  was  performing  correctly  but 
inappropriately,  given  contextual  or  environmental  factors.  In 
the  case  of  a  contextual  error,  the  robot  cannot  recognize  such  a 
malfunction,  so  the  operator  must  take  notice  of  it;  one  example 
may  be  failed  navigation,  in  which  the  robot  would  need  to  be 
redirected  to  a  more  optimal  path.  Participants  were  required 
to  monitor  and  recharge  robots'  batteries,  attend  to  targets,  and 
detect  and  redirect  navigation  errors.  Operator  performance  var- 
ied depending  upon  the  cognitive  demands  of  the  various  tasks. 
Degradation  was  found  to  come  from  "event  time-line  conflicts" 
and  attentional  limitations,  the  latter  of  which  were  most  preva- 
lent in  the  contextual  error  identification  and  redirecting  task. 

In  Trouvain  and  Wolf  [73],  participants  used  either  2,  4,  or 
8  simulated  UGVs  to  perform  inspection  tasks  at  the  predes- 
ignated  inspection  locations.  Results  showed  that  participants' 
overall  task  performance  (number  of  inspections  completed)  im- 
proved from  the  two-robot  condition  to  the  four-robot  condition, 
but  not  from  the  four-robot  to  eight-robot  condition.  However, 
their  data  also  showed  that  when  the  number  of  robots  increased 
from  2  to  4,  the  average  inspection  delay  (amount  of  time  robots 
spent  on  waiting  for  human  inspection  after  reaching  the  inspec- 
tion locations)  also  more  than  doubled.  In  fact,  it  was  found  that 
participants  had  difficulties  keeping  more  than  two  robots  ac- 
tive at  the  same  time.  The  subjective  workload  data  also  showed 
that  participants  experienced  slightly  higher  workload  when  the 
number  of  robots  increased  from  2  to  4;  however,  they  experi- 
enced significantly  higher  workload  when  the  number  of  robots 
was  8. 

Wang  and  Lewis  [70]  explored  operator  demands  of  au- 
tonomous robot  teams  versus  manually  controlled  robot  teams  in 
a  building  search  task.  In  this  within- subjects  experiment,  three 
robots  were  controlled  serially,  so  only  the  selected  robot  would 
accept  commands.  Participants  controlled  waypoint  movements, 
manually  teleoperated  the  robots,  and  operated  a  camera  (pan- 
ning and  tilting  it  to  achieve  an  optimal  view).  Waypoint  and 
teleoperation  control  improved  in  the  autonomous  condition, 
but  camera  control  did  not  differ.  Participants  also  switched 
control  between  the  three  robots  more  frequently  in  the  au- 
tonomous mode  than  in  the  manual  control  mode.  Results  fur- 
ther showed  that  autonomous  cooperation  between  robots  aided 
operator  performance.  Wang  and  Lewis  believed  that  this  was 
because  their  search  task  consisted  of  several  subtasks,  which 
participants  had  to  interact  with  during  the  windows  of  their 
"neglect  time"  (see  [57]).  Switching  control  between  the  robots 
more  frequently  (as  in  the  autonomous  mode)  gave  participants 
more  time  to  finish  the  subtasks  involved  [70]. 

In  a  follow-on  study,  Lewis  and  coworkers  [71]  investigated 
operators'  performance  when  they  controlled  4,  8,  or  12  robots 
(within- subject  variable)  for  a  victim-search  task.  The  results 
showed  that,  in  the  full-task  control  condition  (participants  im- 
plemented waypoints  for  the  robots  and  controlled  the  cam- 
eras), operators  performed  better  (found  more  victims)  with 
eight  robots  compared  with  4  and  12  robots.  Operators'  per- 
ceived workload,  however,  increased  monotonically  with  num- 
ber of  robots.  Wang  et  al  [71],  therefore,  concluded  that  some- 
where between  8  and  12  robots  seemed  to  be  the  limit  for  direct 


human  control.  The  authors  also  suggested  that  automation  of 
navigation-related  tasks  (e.g.,  path  planning)  seemed  to  be  more 
important  than  "efforts  to  improve  automation  for  target  recog- 
nition and  cueing"  [71,  p.  146]  in  the  context  of  controlling  a 
large  team  of  robots. 

Overall,  the  findings  reviewed  in  this  section  seem  to  support 
Olson  and  Wood's  model  [74]  that  FO  can  be  as  high  as  around 
8-9  robots  in  less  demanding  tasking  environments;  however, 
in  more  challenging  conditions,  human  operators  can  only  su- 
pervise around  4-5  robots  effectively  simultaneously.  Opera- 
tor's performance  can  be  enhanced  by  intelligent  coordination 
among  the  robots  and,  according  to  Lewis  and  coworkers  [71], 
automating  navigation-related  tasks  particularly  benefits  opera- 
tor's performance. 


V.  Supervisory  Control  of  Heterogeneous  Unmanned 
Vehicles  (UAVs  and  UGVs) 

Future  warfare  employing  UVs  may  need  to  integrate  infor- 
mation from  multiple  platforms,  potentially  from  both  aerial 
and  ground  sources.  UAVs  generally  provide  exocentric  views 
of  the  problem  space  (i.e.,  the  battlefield)  while  UGVs  present 
viewpoints  that  are  egocentric  and  immersed  in  the  environment. 
The  ideal  view  depends  on  the  task;  overall  awareness  and  pat- 
tern recognition  are  optimized  by  exocentric  views  whereas  the 
immediate  environment  is  often  viewed  better  egocentrically. 
Displays  for  integrating  information  from  different  frames  of 
reference  (FOR;  e.g.,  exocentric  and  egocentric)  present  poten- 
tial human  performance  issues  that  need  to  be  carefully  eval- 
uated [28].  Research  has  shown  that  integrating  information 
across  egocentric  and  exocentric  views  can  be  challenging  for 
the  operator  [77],  [78],  especially  for  those  with  poor  spatial 
ability  [79].  Essentially,  dual  displays  with  both  FOR  require 
effective  scanning  of  the  displays  and  integrating  information 
from  two  different  perspectives  to  form  an  accurate  assessment 
of  the  situation.  Furthermore,  operators  may  be  susceptible  to 
a  saliency  effect  and  anchoring  heuristic/bias  [28].  In  other 
words,  salient  information  on  one  display  may  catch  most  of 
the  operator's  attention,  and  the  operator  may  form  an  inaccu- 
rate judgment  because  information  from  the  other  sources  are 
not  properly  attended  to  and  integrated.  In  Thomas  and  Wick- 
ens  [28],  participants  were  found  to  tunnel  their  attention  into 
the  egocentric  view  to  the  exclusion  of  information  from  the 
exocentric  view. 

Chen  et  al  [4]  simulated  a  command  vehicle  environment 
and  had  the  participants  perform  a  target  designation  task  with  a 
semiautonomous  UAV,  a  semiautonomous  UGV,  teleoperating 
a  UGV,  or,  in  the  mixed  condition,  controlling  all  three  assets. 
They  found  that  when  the  operator  could  use  all  three  assets, 
they  tended  to  ignore  the  equally  efficacious  UGV  and  relied  on 
the  UAV.  This  agreed  with  other  U.S.  Army  sponsored  research 
using  gaming  technology  showing  the  counterintuitive  result 
that  even  when  participants  performed  better  with  UGVs,  they 
still  preferred  UAVs  [80].  This  may  indicate  a  generalized  pref- 
erence for  the  gods-eye  exocentric  view  afforded  by  the  UAVs 
in  comparison  to  the  egocentric  views  obtained  from  the  UGVs. 
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Billman  and  Steinberg  [81]  described  a  set  of  HRI  perfor- 
mance metrics  for  evaluation  of  mixed-initiative  heterogeneous 
robots  (i.e.,  5-10  air,  sea,  and  undersea  UVs).  They  found  that 
these  metrics  were  useful  in  understanding  collaboration  be- 
tween human  operators  and  heterogeneous  robots.  The  metrics 
included:  planning  time,  task  time,  S  A,  operator  workload,  oper- 
ator assessment  of  the  usability  of  the  systems,  and  mental  model 
mapping  (between  reality  and  the  operator's  temporal  and  spa- 
tial mental  model  of  the  system  state).  Lessons  learned  from  a 
series  of  human-in-the-loop  experiments  were  documented  and 
recommended  modifications  of  the  metrics  were  also  presented. 

In  summary,  the  findings  reviewed  in  this  section  suggest  that 
the  effectiveness  of  supervisory  control  of  heterogeneous  UVs 
may  be  affected  by  factors  such  as  operator's  preference  of  view- 
point, attentional  tunneling  phenomenon  induced  by  egocentric 
perspective,  and  individual  differences  in  spatial  ability  (which 
influences  how  well  the  information  from  different  FORs  can 
be  integrated).  In  the  following  section,  we  review  innovative 
techniques  and  technologies  designed  to  enhance  operator  per- 
formance of  supervisory  control  of  UVs. 

VI.  Interface  Designs  for  Supervisory  Control  of 
Multiple  Robots 

When  conducting  supervisory  control  research,  the  interface 
design  must  support  effective  interactions  and  provide  good 
usability,  or  the  design  itself  may  affect  perceived  workload, 
and  overall  performance.  For  instance,  Cummings  [50]  found 
that  intermittent  messages  from  a  chat  communication  tool  in- 
terrupted primary  task  performance.  This  finding  demonstrates 
how  the  presentation  of  alerts  and  alarms  can  be  important  in 
both  experimental  and  "real-world"  system  interfaces,  as  alerts 
and  alarms  may  impact  overall  operator  performance  in  unex- 
pected and  negative  ways  if  they  are  designed  as  independent 
features  of  the  system.  Some  methodologies  for  interface  design 
have  been  investigated  in  the  context  of  complex  supervisory 
control  tasks,  including  abstraction  hierarchy  and  cognitive  task 
analysis  [82],  [83].  For  example,  Linegang  et  al.  [82]  appHed 
the  abstraction  hierarchy  approach  to  design  a  novel  system. 
This  approach  allows  an  interface  designer  to  decompose  a  sys- 
tem into  subcomponents  that  may  each  be  analyzed  in  terms  of 
how  to  best  apply  automation  to  best  aid  the  user.  Additionally, 
Nehme  et  al.  [83]  examined  how  cognitive  task  analysis  could 
be  mixed  with  SA  criteria  to  determine  a  list  of  requirements 
for  interfaces  of  future  technologies. 

Olson  and  Wuennenberg  [84]  presented  a  list  of  recom- 
mended user-interface-design  guidelines  for  supervisory  control 
of  UAVs.  The  recommendations  included  the  following. 

1)  Automation  behavior  (e.g.,  system  status/mode,  system 
goals,  and  flight  control  functions)  should  be  highly  visible 
to  the  operator  (rationale:  operators  tend  to  be  "out  of 
the  loop"  when  systems  are  highly  automated  and  they 
often  find  it  difficult  to  understand  system  behavior  due  to 
system  complexity,  coupling,  and  autonomy). 

2)  It  should  be  easy  for  the  operator  to  extract  meaning  from 
the  display  quickly — minimize  information  access  costs 
by  highlighting  relevant  information,  integrating  dimen- 


sions and  displaying  information  in  appropriate  formats 
(rationale:  operators  often  find  it  difficult  to  understand 
system  behavior  due  to  system  complexity,  coupling,  and 
autonomy). 

3)  Display/highlight  projected  changes  and  predicted  infor- 
mation based  on  operator  inputs  and  direct  operator  at- 
tention to  the  relevant  areas  (rationale:  changes  to  system 
behavior  may  be  difficult  for  the  operator  to  visualize  or 
detect). 

4)  Provide  a  quick  and  easy  way  to  reinstruct  automated 
systems  (rationale:  highly  automated  systems  tend  to  be 
difficult  to  reprogram  and  manual  control  may  not  be  an 
option). 

In  a  recent  study.  Nam  et  al.  [85]  applied  human-computer 
interaction  principles  to  the  design  of  a  multiagent  system  in- 
terface. Using  a  user-centered  design  approach,  three  types  of 
interfaces  were  designed  and  tested.  Each  interface  satisfied  a 
set  of  design  goals  including: 

1)  one  window  display  of  relevant  information; 

2)  user-friendly  layout; 

3)  displaying  only  essential  information  for  decision  making; 

4)  providing  rapid  user  interaction; 

5)  providing  necessary  and  essential  information  for  dynamic 
role  changing; 

6)  providing  feedback. 

Nam  et  al.  concluded  that  interfaces  for  multiagent  systems 
are  efficient  when  they  are  compact  and  when  they  accentuate 
information  that  aids  in  the  decision  making  process.  Based 
on  the  results,  guidelines  for  multiagent  interface  design  were 
proposed. 

Adaptive  automation  can  be  a  very  useful  tool  to  aid  operators 
with  supervisory  control  tasks  as  it  can  "leverage  the  strengths  of 
humans  and  computers  to  improve  overall  system  performance 
while  mitigating  the  negative  aspects  of  both"  [50,  p.  653].  More 
information  on  adaptive  automation  and  other  potential  user- 
interface  designs  for  supervisory  control  of  unmanned  vehicles 
will  be  discussed  in  Section  F. 

Finally,  within  the  context  of  H-A  teaming,  interface  design 
techniques  should  focus  on  supporting  team  processes  between 
the  human  supervisor  and  the  UVs.  As  mentioned  previously, 
team  processes  can  be  supported  by  identifying  team  behaviors 
that  will  occur  during  the  task.  For  interface  design,  the  goal 
would  be  to  identify  characteristics  of  the  interface  that  could 
support  the  team  processes  that  emerge  from  the  H-A  team's 
behaviors.  For  example,  if  shared  awareness  emerges,  then  the 
interface  should  support  communication  methods  that  will  facil- 
itate a  shared  understanding  among  the  team.  This  approach  to 
interface  design  could  be  referred  to  as  a  team- centered 'mitxfdiCt 
design  approach.  Designing  an  interface  using  a  team-centered 
approach  may  enhance  the  robustness  of  the  interface  and  facil- 
itate the  management  of  workload  through  a  balance  between 
task  workload  and  interaction  workload  [86]. 

It  is  important  to  note  that  designing  an  agent's  interface  that 
supports  team  processes  will  not  alone  lead  to  effective  team  in- 
teractions; both  the  human  and  the  agent  are  jointly  responsible 
for  developing  team  play  [49],  [87].  In  Fan  et  a/.'s  [87]  collab- 
oration model  of  an  H-A  team,  agents  can  work  autonomously 
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until  they  run  into  a  problem,  at  which  time  they  could  query  the 
human.  The  active  role  of  the  human  within  the  H-A  team,  there- 
fore, requires  attributes  that  allow  the  human  to  accept/exploit 
an  agent's  capabilities.  For  example,  an  agent  may  support  coor- 
dination among  team  members,  yet,  whether  the  human  accepts 
the  agent's  attempt  to  coordinate  may  depend  on  the  human's 
trust  in  the  agent.  Trust  may  be  established  through  an  un- 
derstanding of  the  agent's  abilities  and  reliability  ([45];  also 
see  Section  II-B).  Therefore,  developing  team  play  within  H-A 
teams  requires  consideration  of  not  only  the  agent's  capabilities 
and  attributes,  but  also  the  human's  knowledge,  skills,  abilities, 
and  attitudes. 

This  section  reviews  several  innovative  user-interface-design 
techniques  in  the  following  areas:  multimodal  displays  and  con- 
trols, planning  tools,  visualization  tools,  attention  management 
tools,  trust  calibration  tools,  adaptive  automation,  and  intelli- 
gent agent  for  human-robot  teaming.  Several  design  guidelines 
based  on  empirical  studies  to  test  the  effectiveness  of  these  de- 
signs are  also  presented. 

A.  Multimodal  Displays/Controls 

Remote  perception  is  essential  for  effective  teleoperation.  In 
teleoperating  environments,  human  perception  is  compromised 
because  the  natural  perceptual  processing  is  decoupled  from  the 
physical  environment.  This  decoupling  affects  people's  percep- 
tion of  affordances  in  the  remote  scene  and  can  have  a  detri- 
mental effect  on  SA  and,  therefore,  the  overall  effectiveness  of 
the  robotics'  task  performance  [88].  The  performance  degrada- 
tions can  be  further  exacerbated  due  to  less  than  ideal  video 
quality  [89],  [90].  Simpson  et  al.  [91]  proposed  using  a  spa- 
tial audio  display  to  augment  UAV  operator's  SA  and  discussed 
its  utility  for  each  of  the  three  SA  levels.  They  recommended 
that  both  visual  and  spatial  audio  information  should  be  pre- 
sented concurrently.  They  also  suggested  that  presenting  the 
audio  information  spatially  may  enhance  UAV  operator's  sense 
of  presence  (telepresence).  However,  the  link  between  presence 
and  task  performance  has  yet  to  be  established.  Additionally, 
spatial  audio  displays  can  be  ineffective  in  a  noisy  environment 
and  the  user  may  experience  front-back  confusion.  Tactile  dis- 
plays, therefore,  present  a  viable  alternative  to  audio  displays. 
Haas  et  al.  [92]  designed  a  soldier-swarm  display  (the  swarm 
consisted  of  40  simulated  ducted  fan  aircraft)  and  compared  its 
effectiveness  when  information  was  displayed  using  visual,  au- 
ditory, and  tactile  combinations.  The  multimodal  displays  (com- 
pared with  the  visual  display  baseline)  significantly  improved 
Marine  participants'  detection  performance  (i.e.,  reduction  in 
response  times)  and  decreased  their  perceived  workload.  In  an- 
other study,  Gunn  et  al.  [93]  compared  the  effectiveness  of  a 
sensory  display  and  a  cognitive  display  for  presenting  warn- 
ing information  in  a  UAV  control  task.  They  found  that  the 
sensory  display  (which  presented  the  information  by  changing 
the  physical  attributes  of  stimuli)  resulted  in  more  threat  de- 
tections, fewer  FAs,  faster  response  time,  and  lower  perceived 
workload  than  did  the  cognitive  display  (which  required  sym- 
bolic manipulations  to  define  critical  signals).  Gunn  et  al.  [93] 
also  compared  the  utility  of  different  types  of  directional  cueing 


interfaces  (visual,  spatial  audio,  and  haptic)  and  found  no  dif- 
ferences in  their  effectiveness.  For  a  comprehensive  review  of 
multimodal  displays  and  controls  for  robotics  control,  see  [89]. 

B.  Planning  Tools 

Planning  (e.g.,  route-planning  task)  is  a  vital  part  of  HRI. 
It  is  increasingly  common  for  plans  to  be  generated  by  auto- 
mated/intelligent systems  in  mixed-initiative  operations.  How- 
ever, lessons  learned  from  a  U.S.  Naval  Intelligent  Autonomy 
program  indicated  that  human  operators  sometimes  questioned 
the  accuracy  and  effectiveness  of  automated  plans  [82].  Specifi- 
cally, some  human  operators  indicated  that  they  had  difficulties 
understanding  the  rationales  for  some  plans  and  how  the  plans 
were  generated  (as  cited  in  [82,  Billman  et  al. ,  2005]).  Addition- 
ally, some  operators  reported  that  they  had  difficulties  at  times 
when  trying  to  specify  mission  parameters  (e.g.,  goals  and  con- 
straints) in  the  way  required  by  the  automated  planning  system 
(as  cited  in  [82,  Billman  et  al,  2005]).  Furthermore,  real-time 
development  on  the  battlefield  may  require  the  human  operator 
to  change  the  plan  for  the  robot  team  and/or  for  the  individual 
robots.  Therefore,  effective  communication  between  the  human 
operator  and  the  robots  is  critical  in  ensuring  planning  effective- 
ness and  mission  success. 

Research  has  been  conducted  on  ways  to  enhance  human- 
robot  communication  [11].  For  example,  Stubbs  et  al.  demon- 
strated the  effectiveness  of  a  robot  proxy  to  enhance  shared 
understanding  between  the  human  operator  and  the  robot  in  an 
exploration  task.  The  communication  mechanism  was  based  on 
a  common  ground  collaboration  model  and  was  able  to  improve 
the  human  operator  performance  in  the  following  areas:  more 
accurate  plans,  more  efficient  planning  (fewer  times  of  replan- 
ning),  more  efficient  and  faster  task  performance,  and  better 
mental  model  of  the  robots'  capabilities.  The  U.S.  Navy  has 
also  sponsored  a  research  and  development  program  to  design 
a  mission-planning  tool  for  human  operators  controlling  a  het- 
erogeneous group  of  robots  [82].  The  tool,  mission  displays  for 
autonomous  systems  (MiDAS),  uses  an  ecological  approach  to 
reduce  conflicts  between  human  operators  and  the  automated 
planning  system. 

1 )  Schedule  Management  Aid:  When  controlling  multiple 
semiautonomous  UVs,  each  executing  its  own  predetermined 
plan,  the  operator  may  experience  high  workload  when  more 
than  one  UV  needs  his/her  attention  at  the  same  time.  It  is, 
therefore,  beneficial  to  let  the  operator  know  when  potential 
processing  bottlenecks  may  occur  so  they  make  arrangements 
to  mitigate  the  bottleneck  if  necessary.  Cummings  et  al.  [94] 
designed  an  intelligent  schedule  management  aid  and  tested 
its  effectiveness  in  a  simulated  suppression  of  enemy  air  de- 
fenses mission  environment  with  four  UAVs.  The  aid  incorpo- 
rated timeline  displays  that  show  potential  bottlenecks  and  it 
also  presented  potential  scheduling  conflicts  using  configurable 
displays  (see  Fig.  1). 

Overall,  both  the  timeline  and  the  configural  displays  were 
effective.  However,  the  authors  cautioned  that,  in  order  for  the 
displays  to  be  effective,  the  user-interface  designers  need  to 
ensure  the  following. 
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Fig.  1.  Timeline  display  and  configural  display  (right)  (adapted  from  [94], 
with  permission). 


Fig.  2.  (Left  to  right)  Individual  robots,  semitransparent,  and  solid  (adapted 
from  [95],  with  permission). 


1)  The  information's  importance  matches  its  representation 
saUence  (rationale:  overly  salient  representations  may 
cause  the  operator  to  fixate  on  something  not  important 
while  overlooking  more  serious  issues). 

2)  The  aid  presents  useful  solutions  to  emerging  problems 
rather  than  simply  providing  visualizations  of  potential 
problems  (rationale:  Cummings  et  al  showed  that  only 
presenting  visualizations  without  solutions  was  not  more 
effective  than  no  visualizations)  [94,  p.  58]. 

Dorneich  et  al.  [51]  took  a  different  approach  and  used  neu- 
rophysiological  sensors  (EEG)  to  detect  cognitive  workload  in 
a  navigation  task  (walking  along  a  familiar  route)  with  an  in- 
termittent communication  task  and  mathematical  interruption 
task.  Throughout  the  experiment,  EEG  sensors  accurately  eval- 
uated cognitive  workload  about  70%  of  the  time,  and  a  Com- 
munication Scheduler  adapted  task  scheduling  information  by 
informing  the  participant  what  to  focus  on  next.  The  communi- 
cation scheduler  positively  impacted  participants'  performance 
by  rescheduling  their  priorities,  resulting  in  only  a  temporary 
loss  of  SA  for  low-priority  messages  [51]. 

C.  Visualization  Tools 

Humphrey  et  al.  [95]  examined  the  use  of  visualizations 
as  an  aid  for  operators  responsible  for  controlling  a  team  of 
robots.  Previous  studies  examined  color-coding  robot  status  and 
team  associations;  this  study  added  shapes  to  represent  an  in- 
dividual robot  (condition  1),  or  teams  of  robots  (condition  2 — 
semitransparent,  and  condition  3 — solid)  (see  Fig.  2).  Partici- 
pants were  tasked  with  selecting  specific  robots  relative  to  other 
robots  or  environmental  characteristics,  and  to  verbally  confirm 
or  deny  accuracy  of  statements  regarding  a  robot's  status.  In- 
vestigators were  interested  in  the  level  of  operator  engagement, 
as  indicated  by  the  frequency  with  which  participants  utilized 
the  visualizations.  Results  indicated  that  the  semitransparent 
and  solid  visualizations  were  preferred  and  utilized  significantly 
more  than  the  individual  visualization.  Participants  slightly  pre- 
ferred (in  a  post-trial  questionnaire)  the  presentation  of  both  an 
individual  and  semitransparent  visualization  (in  the  same  trial). 
The  authors  plan  to  continue  this  research  with  the  hypothesis 
that  an  operator's  SA  will  increase  when  visualization  tech- 


Fig.  3.  Synthetic  vision  symbology  added  to  simulated  UAV  gimbal  camera 
video,  marking  threat,  landmarks,  areas  of  interest  and  runway  (symbology 
generated  with  LandForm  SmartCamSD,  Rapid  Imaging  Software,  Inc.;  adapted 
from  [96],  with  permission). 


niques  are  applied  to  managing  teams  of  robots;  this  technique 
may  also  help  an  operator  supervise  a  larger  team  of  robots. 

1 )  Augmented  Reality:  Augmented  reality  (AR;  also  known 
as  synthetic  vision)  has  been  found  to  be  an  effective  means  to 
enhance  pilot/UAV  operator  SA  by  portraying  a  more  veridical 
view  of  the  combat  environment  [96]-[99].  For  example,  the 
U.S.  Air  Force  has  identified  several  candidate  synthetic  vision 
overlay  concepts  for  UAV  applications  [96]  (see  Fig.  3).  The 
following  information,  potentially,  can  be  overlaid  graphically 
on  the  streaming  video:  maps  and  other  synthetically  generated 
symbology,  photoimagery,  terrain  elevation,  laser  range  scans, 
past  and  potential  future  robot  paths,  updates  via  networked 
communication  with  other  sources,  and  other  vital  statistical 
data  [96],  [100]-[103].  However,  large  amounts  of  information, 
although  helpful  in  reducing  the  operator's  scanning  effort  by 
providing  more  data  in  a  centralized  area  (e.g.,  the  video),  can 
create  visual  clutter  and  degrade  operator's  information  pro- 
cessing [104].  Thus,  it  is  important  that  a  declutter  capability  be 
provided  so  the  operator  can  customize  the  overlaid  information 
presentation  according  to  the  situation  and  tasks. 

More  research  is  needed  to  determine  the  optimal  approach 
to  decluttering  (i.e.,  global  approach  by  deselecting  classes  of 
information  or  local  approach  by  deselecting  individual  sym- 
bology; [104]).  Additionally,  overlaying  information  on  a  video 
feed  can  potentially  lead  to  cognitive  tunneling,  as  operator's 
attention  can  be  captured  by  the  overlaid  data  while  important 
elements/developments  in  the  video  might  be  overlooked  [99], 
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[105],  [106].  However,  a  more  recent  study  by  lani  and  Wick- 
ens  [107]  indicated  that  the  attentional  tunneling  effect  of  the 
AR  displays  may  not  be  as  pronounced  as  previously  suggested. 
The  tradeoff  between  adding  information  to  the  video  feed  and 
cognitive  tunneling  needs  to  be  more  systematically  evaluated. 
A  list  of  human  factors  issues  with  UAV  AR  systems  and  their 
potential  solutions  can  be  found  in  Calhoun  et  al.  [104]. 

2)  Ecological  Interface  Designs:  Ecological  interface  de- 
sign (EID)  is  a  user-interface-design  technique  that  conveys 
the  constraints  in  the  tasking  environment,  usually  visually 
via  emergent  patterns,  so  the  operator  can  intuitively  perceive 
and  solve  the  problem  [108].  Furukawa  and  Parasuraman  [109] 
demonstrated  that  EID  was  beneficial  for  enhancing  human  op- 
erators' detection  of  automation  errors  as  well  as  their  compre- 
hension of  system  states.  In  their  first  experiment,  Furukawa  and 
Parasuraman  showed  that  human  operators,  using  an  EID  dis- 
play showing  an  emergent  perceptual  feature,  were  able  to  detect 
significantly  more  system  errors  than  when  they  used  a  noninte- 
grated  display  (i.e.,  they  showed  significantly  less  automation- 
induced  complacency).  More  strikingly,  the  operators  were  able 
to  achieve  better  performance  even  though  their  visual  atten- 
tion to  the  EID  display  was  significantly  less,  according  to  an 
eye  movement  analysis,  indicating  that  their  monitoring  was 
more  efficient.  In  the  second  experiment,  Furukawa  and  Para- 
suraman [109]  showed  the  effectiveness  of  an  EID  display  that 
portrays  graphically  the  intention  of  the  automated  system.  Their 
results  showed  that  this  visualization  tool  helped  the  human  op- 
erators to  achieve  a  better  mental  model  of  the  system,  which 
enabled  them  to  make  better  decisions.  In  yet  another  study, 
Furukawa  et  al.  [110]  integrated  the  intention-represented  EID 
display  in  a  partially  automated  process  control  simulation  and 
compared  its  effectiveness  with  that  of  an  EID  display  without 
intention  indicators  of  the  automated  system.  Results  showed 
that  the  intention-represented  EID  display  was  able  to  enhance 
the  operators'  predictions  of  the  actions  and  behaviors  of  the 
automated  system  and  therefore,  was  able  to  improve  the  op- 
erator's action  planning  and  decision  making.  Additionally,  the 
benefits  were  demonstrated  in  novel  scenarios,  suggesting  that 
the  operators  had  a  better  mental  model  of  the  automated  system 
with  the  intention-represented  EID  display  than  with  the  EID 
display  without  the  intention  indicators. 

Cummings  and  Bruni  [111]  designed  a  user  interface  that 
supports  a  single  operator's  ability  to  control  four  UAVs  simul- 
taneously. They  utilized  visualization  techniques  in  designing 
a  tool  that  helps  the  operator  replan  the  mission.  For  example, 
they  used  a  configural  display  to  depict  the  overall  cost  of  the 
revised  plan  (see  Fig.  4). 

Furukawa  [112]  also  incorporated  an  EID  display  into  a 
RoboFlag  simulation  program.  The  results  showed  that  the  EID 
display  enhanced  the  human  operator's  performance  of  super- 
vising a  team  of  robots  by  portraying  essential  functional  infor- 
mation (graphical  representations  of  the  offensive  and  defensive 
functions  for  the  RoboFlag-simulated  game).  A  detailed  review 
of  the  EID — the  empirical  evidence  of  the  efficacy  of  EID,  why 
EID  improves  operator  performance,  how  EID  can  be  lever- 
aged, and  challenges  of  implementing  EID — can  be  found  in 
Vincente  [113]. 


Fig.  4.  Supervisory  control  of  four  UAVs — configural  display  is  the  center 
panel  with  three  green  triangles  indicating  the  costs  of  planned  operations  (image 
adapted  from  [111],  with  permission). 


D.  Attention  Management  Tools  (Interruption  Recovery  Aid) 

When  controlling  multiple  robots  at  the  same  time,  it  is 
inevitable  that  the  operator  will  tend  to  some  aspects  of  the 
environment  (e.g.,  one  of  the  robots)  before  resuming  his/her 
monitoring  of  all  the  robots.  Techniques  that  facilitate  task 
resumption  have  been  proposed  and  tested  in  various  tasking 
environments  [114]-[116].  Some  techniques  focus  on  remind- 
ing the  operator  where  s/he  was  before  the  interruption  [114], 
while  others  present  aids  for  the  operator  to  quickly  review 
what  happened  during  the  interruption  [115],  [116].  Ratwani 
et  al.  [114]  demonstrated  that  simply  by  reducing  the  size  (by 
about  75%)  of  the  window  for  the  interrupting  task  (i.e.,  reduc- 
ing the  occlusion  of  the  primary  task  screen  by  the  interrupting 
task  window),  participants  were  able  to  resume  their  primary 
task  significantly  faster.  Eye-tracking  data  also  showed  that  par- 
ticipants were  more  accurate  at  returning  to  where  they  left  off. 
Other  more  sophisticated  techniques  to  facilitate  recovery  from 
interruptions  have  also  been  developed.  For  example,  St.  John 
et  al.  [116]  discussed  the  utility  of  a  SA  recovery  tool  (named 
CHEX;  see  Fig.  5),  which  displayed  a  textual  event  history  list 
in  a  naval  air  warfare  environment  (monitoring  a  geoplot  of  an 
airspace  which  contained  ownship  and  approximately  50  other 
aircraft). 

St.  John  et  al.  found  that  CHEX  was  more  effective  in  helping 
the  operator  resume  their  task  after  interruptions  (blank  screen 
lasting  either  30  or  120  s)  than  a  video  replay  tool,  which  was 
worse  than  the  baseline  condition  (no  aid).  CHEX  presented 
changes  in  a  table  and  linked  them  to  the  map  when  the  operator 
selected  a  change  from  the  table.  However,  Scott  et  al.  [115]  ar- 
gued that  the  ineffectiveness  of  the  video  replay  tool  shown  in  St. 
John  et  al.  [116]  might  be  improved  if  a  better  design  had  been 
adopted.  Scott  et  al.  [115]  presented  two  types  of  replay  tools — 
one  replayed  the  events  at  a  10  x  real-time  speed  and  the  other 
presented  bookmarks  on  the  event  timelines  and  the  operator 
could  view  the  replay  by  selecting  the  bookmarks  (see  Fig.  6). 
Results  showed  that  both  replay  techniques  were  effective. 
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Fig.  5.  Interruption  recovery  aid — textual  event  history  list  (CHEX — upper 
right)  (adapted  from  [116],  with  permission). 


Fig.  6.    Interruption  assistance  interface  (adapted  from  [115],  with  permission). 

especially  when  the  tasking  environment  was  challenging. 
Based  on  the  results,  the  authors  presented  several  recommended 
design  guidelines  for  interruption  assistance  interfaces. 

1)  Enable  user  control  of  event  replay. 

2)  Provide  visual  summary  of  critical  events. 

3)  Limit  visual  summary  to  goal-related  events. 

4)  Clearly  indicate  relationships  between  past  and  current 
system  state  [115,  p.  703]. 

E.  Trust  Calibration  Tools 

Lee  and  See  [45]  recommended  that  the  capabilities  and  lim- 
itations of  the  automated  systems  be  conveyed  to  the  operator, 
when  feasible,  in  order  for  the  operator  to  develop  appropriate 
trust  and  reliance.  Bagheri  and  Jamieson  [117]  demonstrated 
that  when  operators  were  aware  of  the  context-related  nature  of 
automation  reliability,  their  detection  rate  of  automation  failures 
increased  significantly  without  affecting  their  concurrent  track- 
ing and  system  management  tasks.  The  authors  attributed  this 
improvement  in  performance  to  a  more  effective  attentional  al- 
location strategy.  Rovira  et  al  [34]  investigated  the  differential 
effect  of  automation  reliability  and  different  types  of  automa- 
tion (decision  support  and  information  support)  on  performance 


during  a  command  and  control  task.  Their  results  confirmed  a 
differential  cost  of  automation  unreliability  for  three  forms  of 
decision  automation,  as  compared  with  information  automation 
when  the  overall  automation  reliability  was  at  80%.  At  60% 
overall  automation  reliability,  however,  there  was  a  reduction 
in  performance  for  both  information  and  decision  automation 
during  unreliable  trials.  This  finding  suggests  that  the  type  of  au- 
tomation employed  is  irrelevant  when  automation  reliability  is 
below  a  certain  threshold.  Based  on  their  findings  on  the  differ- 
ent types  of  automation,  Rovira  et  al.  suggested  that  decrements 
in  decision-making  performance  will  be  lower  when  operators 
can  query  the  automation,  inspect  raw  information  sources,  and 
verify  or  negate  the  automated  advice. 

In  another  study,  Seppelt  and  Lee  [118]  designed  a  display, 
based  on  EID  that  portrayed  graphically  the  capabilities  and 
limitations  of  the  adaptive  cruise  control  of  a  vehicle  for  a  driv- 
ing task  in  different  traffic  and  weather  conditions.  They  found 
that  drivers'  reliance  on  the  automation  (i.e.,  cruise  control)  was 
more  appropriate  when  the  display  was  present  than  when  it 
was  not.  Lee  and  See  [45]  developed  the  following  guidelines 
on  designing  systems  that  promote  appropriate  trust  in  the  au- 
tomation. 

1)  Design  for  appropriate  trust,  not  greater  trust. 

2)  Show  the  past  performance  of  the  automation. 

3)  Show  the  process  and  algorithms  of  the  automation  by 
revealing  intermediate  results  in  a  way  that  is  comprehen- 
sible to  the  operators. 

4)  Simplify  the  algorithms  and  operation  of  the  automation 
to  make  it  more  understandable. 

5)  Show  the  purpose  of  the  automation,  design  basis,  and 
range  of  applications  in  a  way  that  relates  to  the  user's 
goals. 

6)  Train  operators  regarding  its  expected  reliability,  the 
mechanisms  governing  its  behavior,  and  its  intended  use. 

7)  Carefully  evaluate  any  anthropomorphizing  of  the  automa- 
tion, such  as  using  speech  to  create  a  synthetic  conversa- 
tional partner,  to  ensure  appropriate  trust  [45,  p.  74]. 

F.  Adaptive  Automation 

Adaptive  systems  were  developed  more  than  50  years  ago  to 
aid  the  operator  by  keeping  performance  constant  as  a  function 
of  task  difficulty  [119].  One  of  the  early  paradigms  used  queuing 
theory  as  an  allocation  system  to  service  aviation  tasks  on  which 
the  operator  was  performing  poorly  during  high  workload  sit- 
uations. IVIore  recently,  adaptive  automation  has  been  proposed 
as  a  technique  to  keep  operators  engaged  in  a  multitasking  sit- 
uation without  overwhelming  them  during  peak  workload  situ- 
ations [31],  [120],  [121].  These  systems,  a  middle  ground  be- 
tween fully  automated  and  manual  control,  provide  a  means  of 
keeping  operators  in  the  loop.  The  basic  concept  consists  of  de- 
cisions rules  for  the  computer  invoking  automation  during  high 
workload  and  for  reinvoking  manual  control  during  lower  work- 
load mission  segments.  The  purpose  of  the  adaptive  automation 
is  to  keep  operator  performance  within  acceptable  levels  while 
maintaining  attentional  focus  on  important  tasks.  Decision  tasks, 
in  particular,  should  be  completely  automated  with  extreme  care 
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because  even  with  perfectly  automated  tasks,  the  operator  may 
lose  SA  if  the  environment  changes  or  unexpected  emergencies 
occur  [31],  [34],  [36].  For  imperfect  automation  the  situation  is 
worse.  Even  for  highly  accurate  automated  aids,  operators  tend 
to  over  rely  on  the  automated  solution,  which  can  cause  even 
obvious  errors  to  be  missed  (i.e.,  automation  paradox;  [34]). 
Consequently,  with  less-accurate  aids,  automation  itself  can  be 
a  detriment  [33].  Therefore,  important  issues  with  adaptive  sys- 
tems such  as  type  of  invocation  process,  switching  rate,  and 
engagement  time  must  be  taken  into  account  in  the  design  of 
these  systems. 

Invocation  rules  (when  to  invoke  automated  or  manual  con- 
trol) can  be  based  on  models,  performance  levels,  physiological 
changes,  or  some  mixture  of  them  [36].  A  number  of  physio- 
logical measures  have  been  used  to  covertly  monitor  operator's 
workload  state:  EEG,  event-related  potential  (ERP),  heart-rate 
variability  (HRV),  functional  MRI  (fMRI),  and  combinations 
of  the  measures  [31],  [122].  Freeman  et  ah  [123]  showed  the 
potential  of  using  an  adaptive  system  for  human  tracking  perfor- 
mance using  three  neurophysiological  EEG  indices  for  automa- 
tion invocation.  To  test  whether  EEG  was  a  good  indicator  of 
performance,  the  authors  compared  two  methods  for  invoking 
automation:  high  arousal  components  of  the  EEG  (high  ratios  of 
the  beta  wave  components  divided  by  theta  and/or  alpha  com- 
ponents) and  components  associated  with  low  arousal  (higher 
ratios  of  alpha  and  theta  wave  components).  As  predicted,  con- 
ditions that  used  EEGs  with  high  beta  ratios  resulted  in  better 
tracking  performance  indicating  that  neurophysiological  indices 
could  provide  a  potential  means  of  covertly  measuring  the  oper- 
ator's arousal  level  for  adaptive  processes.  Milkulka  et  al.  [124] 
showed  similar  patterns  of  performance  for  a  vigilance  task  us- 
ing the  high  beta  weightings  of  the  EEG  to  control  adaptive  event 
rate  procedures.  However,  because  yoked  participants  (adaptive 
processes  not  based  on  the  operator's  EEG)  performed  as  well 
as  participants  in  the  high  beta  ratios  conditions,  it  is  possible 
that  task  difficulty  measures  may  be  as  efficient  as  EEG  indices 
for  invoking  automated  processes.  They  also  reported  that  an 
automation  switching  rate  of  about  15  s  between  conditions  was 
sufficient  for  operators  to  be  able  to  focus  on  the  current  task 
and  to  adjust  to  new  adaptive  conditions. 

Wilson  and  Russell  [125]  developed  a  more  sophisticated 
adaptive  automation  invocation  method  using  multiple  physio- 
logical indices  that  were  performance  weighted  using  a  neural 
network  based  algorithm.  The  adaptive  conditions  resulted  in 
better  performance  than  manual  conditions.  More  importantly, 
the  weights  based  on  the  individual's  past  performance  resulted 
in  better  performance  than  group  weighted  invocation  methods, 
indicating  their  indices  were  sensitive  to  individual  differences. 
All  three  studies  [123]-[125]  showed  that  physiological  indices 
were  potential  invocation  methods  for  adaptive  processes;  they 
also  indicated  that  fairly  short  switching  rates  were  feasible. 
Moreover,  the  Wilson  and  Russell  study  [125]  indicated  that 
the  measures  should  be  tailored  to  individual  differences  (see 
also  [51]  discussed  above).  However,  none  of  these  studies  ac- 
tually showed  an  advantage  for  adaptive  processes  compared  to 
static  automation  nor  did  they  compare  physiological  to  non- 
physiological  indices. 


Parasuraman  et  al.  [126]  demonstrated  the  efficacy  of  adap- 
tive processes  in  a  multitasking  aviation  environment.  There 
were  three  conditions:  static  automation,  model-based  adaptive 
automation  (mission  segments  with  high  task  difficulty),  and 
performance-based  adaptive  automation  (operator  error  rate  de- 
termined invocation).  Manual  control  was  invoked  in  the  mid- 
dle of  the  90-min  experiment  for  the  two  adaptive  conditions. 
The  adaptive  system  did  increase  the  SA  of  the  operators — 
showing  significantly  better  instrument  monitoring  performance 
compared  to  the  static  automated  conditions  for  the  sessions 
immediately  following  the  manual  conditions.  The  authors  in- 
terpreted this  as  indicating  that  putting  operators  in  the  loop, 
even  briefly  during  a  90-min  experiment,  increased  their  aware- 
ness compared  to  automating  a  system  for  the  entire  90  min. 
Also,  the  experiment  demonstrated  that  either  model-based  or 
performance-based  rules  could  be  used  as  invocation  triggers 
for  automation. 

Parasuraman  et  al.  [52]  compared  performance-based  adap- 
tive automation,  static  automation,  and  manual  target  identifi- 
cation using  an  automatic  target  recognition  (ATR)  device  in 
the  automated  conditions.  Their  study  employed  a  multitasking 
environment  involving  targeting  with  UAVs  and  planning  UGV 
routes.  The  trigger  for  adaptive  automation  was  the  number  of 
missed  changes  in  a  situation  map,  which  suggests  that  adap- 
tive processes  can  be  invoked  from  secondary  task  as  well  as 
primary  task  difficulties.  The  results  were  dramatic — SA,  work- 
load, and  performance  on  the  secondary  task  were  all  improved 
by  automation,  but  more  importantly,  performance  gains  for  the 
adaptive  (performance-based)  versus  static  (model-based)  au- 
tomation were  substantial.  Based  on  this  study  and  a  review 
of  the  literature,  Parasuraman  et  al.  suggested  the  following 
guidelines  for  adaptive  automation. 

1 )  Information  displays  should  adapt  to  the  changing  military 
environment. 

2)  Software  should  be  developed  to  allow  the  operator  to 
allocate  automation  under  specified  conditions  before  the 
mission  (as  in  the  Rotorcraft  Pilot's  Associate;  [127]). 

3)  At  least  initially,  adaptive  systems  should  be  evaluated  that 
do  not  take  decision  authority  away  from  the  operator.  This 
can  be  accomplished  in  two  ways: 

a)  an  advisory  asking  permission  to  invoke  automation 
(i.e.,  MBC);  or 

b)  an  advisory  that  automation  will  be  invoked  unless 
overridden  (i.e.,  MBE). 

4)  For  safety  or  crew-protection  situations,  specific  tactical  or 
safety  responses  can  be  invoked  without  crew  permission 
[36,  p.  61]. 

Steinhauser  et  al.  [128]  also  provided  several  design  guide- 
lines for  implementing  adaptive  automation  based  on  their  re- 
view of  empirical  research  on  adaptive  automation  and  aiding 
in  the  past  three  decades.  The  guidelines  are  as  follows. 

1)  Adaptive  function  allocation  to  the  operator  should  be 
used  intermittently.  Intermittent  allocation  can  improve 
performance  in  monitoring  tasks. 

2)  Energetic  human  qualities  should  be  considered  in  design. 
For  example,  degrees  of  challenge  can  be  automatically 
adjusted  with  artificial  tasks. 
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3)  Emotional  requirements  of  the  human  operator  must  be 
considered.  The  human  operator  should  not  feel  unneces- 
sary to  the  system  as  a  whole. 

4)  The  system  should  be  calibrated  to  the  individual  operating 
it.  Individual  differences  factor  into  the  human  operator 
portion  of  a  human-system  pairing  and  thus  should  be 
incorporated  into  the  design. 

5)  Task  transformation  should  be  used  to  simplify  tasks  for 
operators.  A  task  that  is  partitioned  and  transformed  can 
be  handled  piecemeal  instead  of  as  a  whole. 

6)  The  environmental  context  of  the  system  should  be  used 
to  determine  allocation.  Environmental  stressors  such  as 
heat,  vibration,  and  gravitational  force  affect  human  per- 
formance and  should  be  addressed. 

7)  Tasks  should  be  partitioned  when  both  the  human  and  the 
system  can  contribute  effectively.  A  true  human-system 
collaboration  operates  as  a  pairing  instead  of  a  dichotomy 
of  effort.  Performance  is  improved  when  the  most  effective 
attributes  of  each  part  are  employed. 

8)  Adaptation  should  be  controlled  by  the  system  but  be  open 
to  human  intervention  when  the  system  fails  to  recognize 
new  conditions  or  demands.  In  order  to  reduce  task  load 
on  the  human  operator  and  improve  general  performance, 
the  system  should  allocate  tasks.  To  improve  satisfaction 
and  motivation,  the  human  operator  should  retain  control, 
or  perceived  control,  of  the  system  [128,  p.  9]. 

Hou  et  al.  developed  an  intelligent  adaptive  system  for  con- 
trolling multiple  UAVs  and  demonstrated  that  the  system  sig- 
nificantly reduced  operators'  workload,  improved  their  SA,  and 
facilitated  the  operators'  ability  to  work  under  high  time  pres- 
sure [129].  Hou  et  al.  [130]  also  presented  a  framework,  based 
on  their  review  of  more  than  200  papers,  for  designing  intelligent 
adaptive  systems.  A  number  of  useful  guidelines  for  designing 
intelligent  adaptive  systems  were  provided  in  this  paper. 

Adaptable  systems  and  adjustable  automation  are  two  re- 
lated schemas  for  combining  automated  and  manual  control. 
Adaptable  systems  allow  the  operator  to  elicit  preprogrammed 
behaviors  during  system  use  similar  to  the  way  a  coach  would 
send  in  various  "plays"  from  a  playbook  during  a  football  game. 
The  advantage  of  adaptable  systems  is  the  introduction  of  more 
flexible  roles  for  supervisory  control  that  allows  the  operator 
to  command  complex  behaviors  without  requiring  complicated 
communication  protocols  [131].  Adjustable  automation  allows 
operators  to  fix  the  level  of  system  autonomy,  allowing  them  to 
decide,  depending  on  the  changing  situation,  how  much  decision 
authority  to  reserve  for  themselves,  and  how  much  autonomy 
to  delegate  [132],  [133].  The  best  type  of  adaptive  system  obvi- 
ously depends  on  the  task  environment,  the  amount  of  complex- 
ity required,  and  the  ability  of  the  operator  to  delegate  autonomy 
during  high  workload  environments.  However,  it  should  be  kept 
in  mind  that  delegation  and  choosing  an  appropriate  autonomy 
level  can  be  very  high- workload  decisions  in  themselves.  Hardin 
and  Goodrich  [134]  demonstrated  that  a  mixed-initiative  control 
scheme  (allowing  the  human  operator  and  the  intelligent  robotic 
agents  to  jointly  decide  the  appropriate  level  of  autonomy)  re- 
sulted in  better  overall  performance  than  did  adaptive  autonomy 
and  adjustable  autonomy  in  a  simulated  target  search  task.  The 


authors,  however,  caution  that  mixed-initiative  control  schemes 
should  meet  the  following  requirements  in  order  to  be  effective. 

1)  The  human  operator  and  the  agents  should  have  comple- 
mentary abilities. 

2)  Agents  should  have  the  capabilities  to  progress  without 
waiting  for  commands  from  the  human  operator. 

3)  The  human  operator  must  be  able  to  interact  with  multiple 
agents  efficiently  and  simultaneously  [134,  p.  172]. 

The  next  section  further  discusses  teaming  between  human 
and  intelligent  agents. 

G.  Intelligent  Agent  and  Human-Robot  Teaming 

The  necessity  for  more  powerful  intelligent  agents  that  can  in- 
teract with  human  operators  in  increasingly  sophisticated  ways 
will  require  that  current  performance  improving  techniques  and 
technologies  be  augmented  with  techniques  and  technologies 
that  facilitate  effective  H-A  team  interactions  [135]-[138].  In 
recent  years,  researchers  have  designed  agents  that  support  team 
behaviors.  Dias  etal.  [139],  for  example,  presented  the  architec- 
ture of  a  dynamically  formed  Human-robot  team  that  performs 
coordinated  tasks.  The  architecture  was  developed  based  on  the 
vision  that  H-A  teams  will  dynamically  form  to  solve  com- 
plex tasks  by  coordinating  their  complementary  capabilities. 
The  architecture  included  TraderBot  [140],  TeamTalk  [141], 
and  Plays  [142].  These  components  were  used  to  support  the 
development  of  coordination  among  both  H-A  teams  and  Agent- 
Agent  teams. 

TraderBot  supports  team  coordination  by  allocating  tasks  to 
other  agents  using  a  market-based  coordination  approach.  This 
approach  is  analogous  to  and  is  described  within  the  context  of 
market  trading  [139].  Market-based  approaches  can  distribute 
much  of  the  planning  and  execution  of  an  operation  over  the 
team,  gather  information  about  the  team,  and  distribute  re- 
sources all  within  a  team  aware  context.  Within  the  architec- 
ture described  by  Dias  et  al,  each  team  member  is  assigned 
an  interface  agent,  or  trader  (see  [143]).  This  agent  is  the  team 
members'  interface  to  the  market  and  simplifies  the  communi- 
cation process  among  human  and  agent  team  members.  The  use 
of  TeamTalk,  a  multiagent  multimodal  dialog  system,  further 
simplifies  the  communication  between  humans  and  agents.  Fi- 
nally, the  architecture  supports  shared  environment  awareness 
by  broadcasting  all  user  input  to  all  robots  on  the  team.  There- 
fore, coordination,  communication,  and  shared  awareness  are 
all  supported  within  the  architecture. 

Yen  et  al.  [144],  presented  research  on  agents  that  can  support 
the  development  of  shared  mental  models.  These  researchers 
discussed  the  use  of  the  collaborative  agents  for  simulating 
teamwork  (CAST)  model,  which  enables  agents  and  humans 
to  anticipate  potential  information  needs  and  proactively  ex- 
change information  between  team  members.  Component  of  the 
CAST  model  were  tested  in  a  simulated  battlefield  over  two  ex- 
periments. The  ability  of  CAST  to  support  communication  and 
decision  making  was  investigated.  For  the  scenario  used  in  the 
first  study,  the  human  was  considered  a  virtual  agent  and  invis- 
ible to  other  team  agents.  Human  team  members  could  affect 
team  activities  by  directly  adjusting  their  agent's  behaviors  and 
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strategies.  For  this  team,  the  decision-making  process  was  sup- 
ported by  the  interactions  (e.g.,  communication)  between  the  hu- 
man and  the  agents;  the  human  provided  domain  expertise,  while 
the  agents  gathered  relevant  information  by  collaborating  with 
other  agents.  In  experiment  one,  the  use  of  a  CAST-supported 
communication  model  (i.e.,  decision-theoretic)  was  compared 
to  a  non-C  AST- supported  communication  model.  Results  of  the 
simulation  indicated  that  using  the  decision-theoretic  approach, 
which  analyzes  the  cost  benefit  ratio  of  communicating  with 
other  team  members  versus  not  communicating  with  other  team 
members,  produced  better  decision  making  than  when  team 
members  communicated  every  time. 

In  the  simulated  scenario  used  in  the  second  experiment,  hu- 
mans had  an  assistant  agent  that  was  invisible  to  the  other  team 
members.  The  assistant  agent  supported  the  human  by  fusing 
and  filtering  information  received  from  other  agents,  tracking 
the  human  team  member's  mental  model,  and  tracking  the  men- 
tal model  shared  among  the  team.  Here,  both  the  interactions 
between  the  human  and  the  agents,  and  the  interaction  between 
the  human-  and  the  assistant-agent-supported  decision  making. 
For  this  experiment,  the  use  of  information  fusion  provided  by 
the  assistant  agent  was  compared  to  not  using  information  fusion 
(i.e.,  no  assistant  agent)  by  assessing  the  accuracy  of  decision 
making,  particularly  when  the  human's  cognitive  capacity  was 
taxed.  The  results  indicated  that  the  simulated  fusion  team,  in 
general,  performed  better  than  the  nonfusion  team.  Furthermore, 
the  disparity  in  performance  between  the  two  simulated  teams 
increased  as  cognitive  capacity  decreased. 

The  results  of  the  two  experiments  by  Yen  et  al.  [144]  sug- 
gest that  using  Collaboration  models  such  as  CAST  to  form 
H-A  teams  may  lead  to  better  performance.  However,  future 
research  should  determine  whether  the  increase  in  performance 
was  a  result  of  the  CAST  approach  facilitating  team  cognition. 
For  example,  did  the  decision-theoretic  approach  support  team- 
decision-making  processes  and/or  did  fusion  lead  to  an  increase 
in  shared  mental  models  among  team  members?  Investigating 
these  questions  along  with  investigating  factors  that  may  in- 
fluence an  operator's  decision  to  accept  an  agent's  attempt  to 
establish  team  cognition  just  begins  to  open  the  door  to  the 
design  of  future  human-robot  team  systems. 

The  missing  element  in  current  research  is  the  development 
of  a  true  peer  relationship  between  humans  and  artificially  in- 
telligent entities.  Work  cited  suggest  the  underpinning  of  peer 
relationship  in  trust,  artificial  animal  surrogates,  advanced  in- 
terfaces, collaborative  behaviors,  supervisory  control  allocated 
to  robotic  entities,  and  even  shared  ethical  values.  Whereas  this 
ongoing  research  all  contributes  to  a  shared  S  A,  the  most  impor- 
tant element  of  a  shared  linguistic  awareness  is  still  in  a  nascent 
state.  Natural  language  interfaces  by  themselves  are  not  suffi- 
cient for  peer  relationships  which  require  more  robust  shared 
cognitive  architectures  that  allow  the  artificial  entity  to  infer  in- 
tent, generalize  instructions,  and  participate  in  a  problem  solv- 
ing dialogue  in  novel  situations.  Intelligent  agents  in  a  mature 
state  that  can  operate  as  peers  in  truly  complex  environments 
will  most  likely  not  be  available  in  the  short  term.  However, 
defining  the  characteristics  of  peer  relationships  is  an  important 
requisite  for  understanding  the  boundaries  of  the  relationship. 


For  example,  delineating  human-to-human  problem  solving  in 
environments  of  interest  will  help  to  isolate  cognitive  factors 
necessary  to  communicate  in  novel  environments.  Investigating 
cognitive  architectures  in  simpler  environments  will  help  mea- 
sure what  variables  influence  peer  synergy  for  intent  inferencing 
and  two-way  dialogues.  There  is  no  requirement  for  intellectual 
equality  between  human  and  artificial  agents;  the  important  is- 
sue is  to  understand  what  factors  are  necessary  for  peer  problem 
solving  in  truly  interesting,  real-world  environments. 

VII.  Conclusion 

This  paper  examined  human  performance  issues  in  super- 
visory control  of  robotic  vehicles  and  reviewed  user-interface 
solutions  that  could  potentially  address  those  issues.  As  robotics 
become  increasingly  prevalent  in  both  military  and  civilian  op- 
erations, it  is  important  to  understand  HRI  and  its  associated 
limitations  as  well  as  potentials.  In  the  foreseeable  future,  it  will 
be  more  common  for  humans  to  work  with  robots  as  a  team 
to  perform  tasks  that  humans  cannot  realistically  accomplish 
alone.  Research  programs  such  as  the  U.S.  Army's  SOURCE 
Army  Technology  Objective  (ATO)  were  also  initiated  to  ex- 
plore how  to  enhance  operator  performance  by  employing  ad- 
vanced technologies  and  user-interface-design  concepts.  These 
solutions,  and  other  innovative  user-interface  designs  reviewed 
in  this  report,  can  hopefully  improve  HRI  and  therefore  reduce 
the  current  challenges  of  operators'  robotic  control  tasks. 
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Objective:  A  military  multitasking  environment  was 
simulated  to  examine  the  effects  of  an  intelligent  agent, 
RoboLeader,  on  the  performance  of  robotics  operators. 

Background:  The  participants'  task  was  to  manage  a 
team  of  ground  robots  with  the  assistance  of  RoboLeader, 
an  intelligent  agent  capable  of  coordinating  the  robots  and 
changing  their  routes  on  the  basis  of  battlefield  developments. 

Method:  In  the  first  experiment,  RoboLeader  was 
perfecdy  reliable;  in  the  second  experiment,  RoboLeader's 
recommendations  were  manipulated  to  be  either  false-alarm 
prone  or  miss  prone,  with  a  reliability  level  of  either  60%  or 
90%.  The  visual  density  of  the  targeting  environment  was 
manipulated  by  the  presence  or  absence  of  friendly  soldiers. 

Results:  RoboLeader,  when  perfectly  reliable,  was  helpful 
in  reducing  the  overall  mission  times. The  type  of  RoboLeader 
imperfection  (false-alarm  vs.  miss  prone)  affected  operators' 
performance  of  tasks  involving  visual  scanning  (target 
detection,  route  editing,  and  situation  awareness).  There  was 
a  consistent  effect  of  visual  density  (clutter  of  the  visual  scene) 
for  multiple  performance  measures.  Participants'  attentional 
control  and  video  gaming  experience  affected  their  overall 
multitasking  performance.  In  both  experiments,  participants 
with  greater  spatial  ability  consistently  outperformed  their 
low-spatial-ability  counterparts  in  tasks  that  required  effective 
visual  scanning. 

Conclusion:  Intelligent  agents,  such  as  RoboLeader, 
can  benefit  the  overall  human-robot  teaming  performance. 
However,  the  effects  of  type  of  agent  unreliability,  tasking 
requirements,  and  individual  differences  have  complex 
effects  on  human-agent  interaction. 

Application:  The  current  results  will  facilitate  the 
implementation  of  robots  in  military  settings  and  will  provide 
useful  data  to  designs  of  systems  for  multirobot  control. 

Keywords:  human-robot  interaction,  supervisory  con- 
trol, intelligent  agent,  military,  imperfect  automation,  indi- 
vidual differences,  spatial  ability,  attentional  control,  gaming 
experience 
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INTRODUCTION 

Robots  are  being  used  more  frequently  in 
military  operations,  and  the  tasks  they  are  being 
used  for  are  evolving  in  complexity.  In  the 
future  battlefield,  soldiers  may  be  given  multi- 
ple tasks  to  perform  concurrently,  such  as  navi- 
gating a  robot  while  conducting  surveillance, 
maintaining  local  security  and  situation  aware- 
ness (SA),  and  communicating  with  fellow 
team  members.  The  possibility  of  a  robotic 
battlefield  creates  a  number  of  human  factors  as 
well  as  ethical  issues  related  to  nonhuman  intel- 
ligence conducting  combat  missions  (Barnes  & 
Evans,  2010;  Singer,  2010). 

A  potential  issue  is  that  the  proliferation  of 
intelligent  systems  could  easily  overwhelm  the 
human  operators'  ability  to  adequately  super- 
vise these  systems  (Chen,  Durlach,  Sloan,  & 
Bowens,  2008;  Schurr,  2007).  As  the  size  of  the 
robot  team  increases,  human  operators  may  fail 
to  maintain  adequate  SA  when  their  attention  is 
constantly  switching  between  the  robots;  cogni- 
tive resources  may  also  be  overwhelmed  by  the 
numerous  intervention  requests  from  the  robots 
(Lewis  et  al.,  2010).  Research  shows  that  auton- 
omous cooperation  between  robots  can  aid  the 
performance  of  the  human  operators  and 
enhance  the  overall  human-robot  team  perfor- 
mance (Lewis  et  al.,  2010;  Schurr,  2007). 
However,  human  operators'  involvement  in 
mixed-initiative  teams  will  still  be  required  for 
the  foreseeable  future,  especially  in  situations 
involving  critical  decision  making.  Human 
operators'  decision  making  may  be  influenced 
by  "implicit  goals"  that  the  robots  are  not  aware 
of  (i.e.,  are  not  programmed  into  the  behaviors 
of  the  robots;  Linegang  et  al.,  2006)  and  by  real- 
time developments  on  the  battlefield  that  may 
require  the  human  operator  to  change  plans  for 
individual  robots  or  the  entire  robotic  team. 
Effective  communication  between  the  human 
operator  and  robots,  therefore,  becomes  critical 
in  ensuring  mission  success. 
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Past  research  has  demonstrated  the  effective- 
ness of  a  robot  proxy  to  enhance  shared  under- 
standing between  the  human  operator  and  the 
robot  in  an  exploration  task  (Stubbs,  Wettergreen, 
&  Nourbakhsh,  2008).  Other  research  and  devel- 
opment efforts  more  related  to  supervisory  control 
of  military  robotics  include  the  U.S.  Army's 
Playbook  program  and  the  U.S.  Air  Force's 
Vigilant  Spirit  Control  Station  for  management  of 
multiple  unmanned  aerial  systems  (UASes;  Fern 
et  al.,  2011;  Miller  &  Parasuraman,  2007). 

Current  Study 

To  achieve  a  better  balance  of  enhancing 
autonomy  and  capability  while  simplifying 
human-robot  interaction,  a  robotic  surrogate  for 
the  human  operator,  RoboLeader,  was  developed 
under  the  U.S.  Army  Research  Laboratory's 
(ARL)  Director's  Research  Initiative  Program 
(Snyder,  Qu,  Chen,  &  Bames,  2010).  RoboLeader 
is  an  intelligent  agent  that  interprets  an  operator's 
intent  and  issues  detailed  command  signals  to  a 
team  of  robots  of  lower  capabilities.  Instead  of 
directly  managing  each  individual  robot,  the 
human  operator  deals  only  with  a  single  entity, 
RoboLeader.  The  operator  can,  therefore,  better 
focus  on  other  tasks  requiring  his  or  her  attention. 

The  present  research  focuses  on  issues  related 
to  human-agent  teams:  the  dynamics  of  using  an 
agent  as  a  subordinate  supervisor  to  aid  in  multi- 
robot  control  as  a  function  of  mission  task  load 
and  agent  reliability  as  well  as  individual  differ- 
ences among  human  operators.  Specifically,  we 
were  interested  in  understanding  the  effects  of 
using  an  intelligent  agent  as  a  team  member  in 
highly  demanding  multitasking  environments 
when  its  suggestions  were  less  than  perfect  and 
in  understanding  how  individual  differences 
interacted  with  agent  unreliability.  To  date,  there 
has  been  minimal  research  on  supervisory  con- 
trol investigating  all  these  factors  in  one  study. 

We  examined  the  effectiveness  of  RoboLeader 
for  enhancing  the  overall  human-robot  teaming 
performance  in  two  experiments.  In  Experiment 
1,  we  compared  the  operators'  target  detection 
performance  in  the  four-robot  and  eight-robot 
conditions,  with  or  without  RoboLeader.  In 
Experiment  2,  we  investigated  the  effects  of  vari- 
ous reliability  levels  for  RoboLeader  on  operators' 


multitasking  performance;  the  reliability  of 
RoboLeader 's  recommendations  was  manipu- 
lated to  be  either  false-alarm  prone  (FAR)  or  miss 
prone  (MP),  with  a  rehability  level  of  either  60% 
or  90%.  In  both  experiments,  the  effects  of  indi- 
vidual differences  in  spatial  ability,  attentional 
control,  and  gaming  experiences  were  examined. 
In  the  rest  of  this  section,  we  briefly  review  the 
main  theoretical  issues  investigated  in  these 
experiments.  For  a  detailed  review  on  human  per- 
formance issues  related  to  supervisory  control  of 
multiple  robots,  see  Chen,  Bames,  and  Harper- 
Sciarini(2011). 

Imperfect  Automation 

Since  it  is  unlikely  for  any  automated  sys- 
tems to  achieve  100%  reliability  at  all  times, 
the  effects  of  unreliable  automation  on  human 
operator  performance  need  to  be  better  under- 
stood before  these  intelligent  systems  can  be 
implemented.  The  effects  of  imperfect  automa- 
tion are  examined  by  Meyer  (2001,  2004),  who 
suggests  that  FAP  and  MP  alerts  may  affect  the 
use  of  an  automated  system  in  different  ways. 
High  false  alarm  (FA)  rates  were  seen  to  reduce 
the  operator's  response  to  alerts  (i.e.,  compli- 
ance), and  high  miss  rates  reduced  the  opera- 
tor's reliance  on  automated  systems.  Similar 
results  were  reported  in  Dixon  and  Wickens 
(2006).  They  found  that  the  operator's  perfor- 
mance of  the  automated  task  degraded  when 
the  automation  was  FAP  because  of  the  opera- 
tor's reduced  compliance  with  the  automated 
system;  when  the  miss  rate  was  high,  the  opera- 
tor's performance  of  the  concurrent  task  was 
affected  more  than  the  automated  task  because 
the  operator  allocated  more  visual  attention  to 
monitor  the  automated  task. 

McCarley  (2007)  showed  that  FAP  automa- 
tion hurt  performance  more  on  the  automated 
task  than  did  MP  automation  ("cry  wolf  effect) 
and  hurt  performance  at  least  as  much  as  MP 
automation  on  the  concurrent  task  (pp.  570- 
571).  Finally,  Wickens  and  Dixon  (2007)  dem- 
onstrated that  when  the  reliability  level  is  below 
approximately  70%),  operators  will  often  ignore 
the  alerts.  In  their  meta-analytic  study,  Wickens 
and  Dixon  found  that  "a  reliability  of  0.70  was 
the  'crossover  point'  below  which  unreliable 
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automation  was  worse  than  no  automation  at 
all"  (p.  201).  In  our  first  experiment,  which 
serves  as  a  baseline  investigation  of  the  utility 
of  RoboLeader,  a  perfect  reliability  level  was 
implemented.  In  the  second  experiment,  we 
manipulated  both  RoboLeader 's  error  type  and 
reliability  level  to  systematically  examine  the 
effects  of  automation  imperfections  on  the  over- 
all human-robot  team  performance. 

Individual  Differences 

In  the  current  study,  we  also  sought  to  evalu- 
ate whether  individual  differences  in  spatial 
ability,  attentional  control,  and  video  gaming 
experience  might  affect  the  operator's  perfor- 
mance. Spatial  ability  (SpA)  has  been  found  to 
be  a  significant  factor  in  certain  visual  display 
domains  (Stanney  &  Salvendy,  1995),  multi- 
tasking involving  flight  asset  monitoring  and 
management  (Morgan  et  al.,  2011),  virtual  envi- 
ronment navigation  (Chen,  Czerwinski,  & 
Macredie,  2000),  target  search  task  (Chen, 
2010;  Chen  et  al.,  2008;  Chen  &  Joyner,  2009; 
Chen  &  Terrence,  2008,  2009),  and  robotics  task 
performance  (Cassenti,  Kelley,  Swoboda,  & 
Patton,  2009;  Lathan  &  Tracey,  2002).  U.S.  Air 
Force  scientists  (Chappelle,  McMillan,  Novy,  & 
McDonald,  2010;  Chappelle,  Novy,  Randall,  & 
McDonald,  2010)  interviewed  53  subject  matter 
experts  about  abilities  that  were  critical  to  effec- 
tive performance  of  UAS  control  tasks  in  terms 
of  piloting  and  sensor  operations;  SpA  was  iden- 
tified as  an  important  factor  for  both  tasks.  Our 
previous  research  showed  that  individuals  with 
greater  SpA  exhibited  more  effective  visual 
scanning  and  target  detection  performance 
(Chen,  2010;  Chen  et  al.,  2008;  Chen  &  Joyner, 
2009;  Chen  &  Terrence,  2008,  2009). 

In  addition  to  spatial  ability,  the  relationship 
between  attentional  control  and  multitasking  per- 
formance was  also  examined.  Attentional  control 
is  defined  as  one's  ability  to  focus  and  shift  atten- 
tion in  a  flexible  manner  (Derryberry  &  Reed, 
2002).  Several  studies  have  shown  that  there  are 
individual  differences  in  multitasking  perfor- 
mance, and  some  people  are  less  prone  to  perfor- 
mance degradation  during  multitasking  conditions 
(Rubinstein,  Meyer,  &  Evans,  2001;  Schumacher 
et  al.,  2001).  There  is  evidence  that  people  with 


better  attention  control  can  allocate  their  attention 
more  flexibly  and  effectively,  and  attention- 
switching  flexibility  can  predict  performance  of 
such  diverse  tasks  as  flight  training  and  driving 
(Bleckley,  Durso,  Crutchfield,  Engle,  &  Khanna, 
2003;  Derryberry  &  Reed,  2002;  Feldman  Barrett, 
Tugade,  &  Engle,  2004;  Kahneman,  Ben-Ishai,  & 
Lotan,  1973).  According  to  a  recent  U.S.  Air 
Force  survey  of  subject  matter  experts  on  the  per- 
formance of  operators  of  UASes  (Chapelle, 
McMillan  et  al.,  2010),  attentional  control  is  one 
of  the  most  important  abilities  that  would  affect 
an  operator's  performance,  since  the  robotics  con- 
trol task  is  inherently  multitasking  (e.g.,  sensor 
manipulation,  tracking,  communication). 

According  to  Feldman  Barrett  et  al.  (2004), 
those  with  lower  attentional  control  tend  to  take 
the  "cognitive  miser"  approach  (i.e.,  conserving 
the  amount  of  cognitive  resources)  when  dealing 
with  complex  information  processing  to  reduce 
the  attentional  control  requirements.  When  deal- 
ing with  automated  systems,  therefore,  it  is  likely 
that  operators  with  different  levels  of  attention 
switching  abilities  may  react  differently  to  unre- 
liable automated  systems.  In  other  words,  opera- 
tors' behaviors  of  compliance  with,  and  reliance 
on,  automation  may  be  altered  by  their  ability  to 
effectively  switch  their  attention  among  the  sys- 
tems. For  example,  the  automation-induced 
complacency  effect  repeatedly  demonstrated  in 
previous  research  (Dzindolet,  Pierce,  Beck, 
Dawe,  &  Anderson,  2001;  Parasuraman  & 
Manzey,  2010;  Thomas  &  Wickens,  2004;  Young 
&  Stanton,  2007)  may  be  more  severe  for  indi- 
viduals with  poor  attentional  control  compared 
with  those  with  better  attentional  control.  This 
phenomenon  has  been  demonstrated  in  Chen  and 
Terrence  (2009)  and  was  further  tested  in  the  cur- 
rent experiment,  which  employed  a  simulation 
environment  considerably  different  from  that  of 
Chen  and  Terrence 's. 

Finally,  the  current  study  sought  to  examine 
the  relationship  between  participants'  video  gam- 
ing experience  and  their  task  performance  as 
well  as  SAof  the  mission  environment.  According 
to  Green  and  Bavelier  (2006)  and  Hubert- 
Wallander,  Green,  and  Bavelier  (2010),  experi- 
enced action  video  game  players,  compared  with 
infrequent  gamers  and  nongamers,  were  found  to 
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Figure  1.  RoboLeader  user  interface. 


perform  significantly  better  on  tasks  that  required 
visuospatial  selective  attention,  multiple  object 
tracking,  rapid  process  of  visual  information  and 
imagery,  and  flexibility  in  attention  allocation. 
Hambrick,  Oswald,  Darowski,  Rench,  and  Brou 
(2010)  also  demonstrated  the  relationship 
between  video  game  experience  and  multitask- 
ing performance.  Therefore,  we  expected  fre- 
quent gamers  would  outperform  infrequent 
gamers  in  our  visually  demanding  task  environ- 
ment in  terms  of  target  and  event  detection  and 
monitoring  of  the  mission  environment. 

EXPERIMENT  1 

In  Experiment  1,  a  military  reconnaissance 
mission  environment  was  simulated  to  examine 
the  effectiveness  of  RoboLeader,  an  intelligent 
agent  capable  of  coordinating  a  team  of  ground 
robots  and  changing  their  routes  on  the  basis  of 
developments  in  the  mission  environment.  The 
participants'  task  was  to  manage  either  four  or 
eight  robots  with  or  without  the  assistance 
of  RoboLeader  while  searching  for  hostile  tar- 
gets via  streaming  video  from  the  robots.  The 
experiment  is  a  mixed-model  design,  with 
RoboLeader  (with  or  without  RoboLeader 


[baseline])  as  the  between-subject  variable  and 
the  number  of  robots  used  in  the  scenario  (four 
vs.  eight)  as  the  within- subject  variable. 

Method 

Participants.  A  total  of  30  individuals  from 
the  Orlando,  Florida,  area  (17  males  and  13 
females,  mean  age  24.7)  participated  in  the 
study.  Participants  were  compensated  $15  per 
hour  for  their  participation  in  the  experiment. 

Simulators.  A  modified  version  of  the 
Mixed  Initiative  Experimental  (MIX)  Testbed 
was  used  as  the  simulator  for  this  experiment 
(Barber,  Davis,  Nicholson,  Finkelstein,  & 
Chen,  2008).  The  Operator  Control  Unit 
(OCU)  of  the  MIX  Testbed  (Figure  1)  was 
modeled  after  the  Tactical  Control  Unit  devel- 
oped under  the  ARL  Robotics  Collaborative 
Technology  Alliance.  RoboLeader  has  the 
capability  of  collecting  information  from  sub- 
ordinate robots  with  limited  autonomy  (e.g., 
collision  avoidance  and  self-guidance  to 
reach  target  locations),  making  tactical  deci- 
sions, and  coordinating  the  robots  by  issuing 
commands,  waypoints,  or  motion  trajectories 
(Snyder  et  al.,  2010).  See  the  Procedure 


144 


section  for  more  details  about  the  user  inter- 
face of  RoboLeader. 

Surveys  and  tests.  A  demographics  question- 
naire was  administered  at  the  beginning  of  the 
training  session.  An  Ishihara  color  vision  test 
(with  nine  test  plates)  was  administered  via 
PowerPoint  presentation.  The  RoboLeader  user 
interface  employed  several  colors  to  display  the 
plans  for  the  robots,  and  normal  color  vision 
was  required  to  effectively  interact  with  the  sys- 
tem. A  questionnaire  on  attentional  control 
(Derryberry  &  Reed,  2002)  was  used  to  evalu- 
ate participants'  perceived  attentional  control 
(PAC).  The  Attentional  Control  survey  consists 
of  21  items  and  measures  attention  focus  and 
shifting.  The  scale  has  been  shown  to  have  good 
internal  reliability  (a  =  .88).  The  Cube  Com- 
parison Test  (Ekstrom,  French,  &  Harman, 
1976)  and  the  Spatial  Orientation  Test  (Gugerty 
&  Brooks,  2004)  were  used  to  assess  partici- 
pants' spatial  ability.  Participants'  perceived 
workload  was  evaluated  with  use  of  the  com- 
puter-based version  of  the  NASA  Task  Load 
Index  questionnaire  (NASA-TLX;  Hart  & 
Staveland,  1988). 

Procedure.  Participants  were  randomly 
assigned  to  the  RoboLeader  group  or  the  base- 
line (no  RoboLeader)  group  before  their  ses- 
sions started.  Before  the  training  session,  they 
completed  the  preliminary  tests  (color  vision 
and  spatial)  and  surveys  (demographic  and 
PAC).  Training,  lasting  about  1  hr,  was  self- 
paced  and  was  delivered  by  PowerPoint  slides 
showing  the  elements  of  the  OCU,  steps  for 
completing  various  tasks,  several  mini  exer- 
cises for  practicing  the  steps,  and  exercises  for 
performing  the  experimental  tasks.  The  partici- 
pants had  to  demonstrate  that  they  could  recall 
all  the  steps  for  performing  the  tasks  without 
any  help.  The  experimental  session  immedi- 
ately followed  the  training  session  and  consisted 
of  two  scenarios,  each  lasting  approximately  30 
min,  in  which  participants  used  their  robotic 
assets  to  locate  20  targets  (i.e.,  10  insurgents 
carrying  weapons  and  10  improvised  explosive 
devices  [lEDs])  in  the  remote  environment. 
There  were  four  robots  available  in  one  scenario 
and  eight  robots  in  the  other  scenario.  The 
order  of  scenarios  was  counterbalanced  across 
participants. 


When  each  scenario  started,  the  robots  began 
by  following  preplanned  routes,  at  which  time 
the  operator's  task  of  monitoring  the  environ- 
ment and  detecting  insurgents  or  lEDs  began 
(by  clicking  on  one  of  the  four  video  thumbnail 
views  highlighted  in  the  color  of  its  route  to 
enlarge  it  into  the  expanded  view  on  the  top  of 
the  screen).  Because  of  the  size  of  the  thumb- 
nails, the  participants  needed  to  enlarge  them 
into  the  expanded  view  to  identify  the  targets. 
The  robots  did  not  have  aided  target  recognition 
capability;  therefore,  the  participants  had  to 
detect  the  10  insurgents  (individuals  dressed  in 
Middle  Eastern  attire  and  carrying  weapons) 
and  10  lEDs  by  themselves. 

For  the  insurgents,  participants  used  their 
computer  mouse  to  click  on  the  targets  to  "lase" 
them  as  soon  as  they  were  detected.  The  "lased" 
insurgents  were  automatically  displayed  on  the 
map.  For  the  lEDs,  the  participants  clicked  on 
the  lED  button  on  the  interface  and  then  marked 
the  locations  of  the  lEDs  on  the  map.  There 
were  friendly  dismounted  soldiers  (individuals 
dressed  in  military  uniform)  and  civilians  (indi- 
viduals dressed  in  Middle  Eastern  attire  without 
weapons)  in  the  simulated  environment  to 
increase  the  visual  noise  for  the  target  detection 
tasks.  The  participants  were  told  that  their 
objective  was  to  finish  reconnoitering  the  area 
using  their  robots  in  the  least  amount  of  time 
possible.  Therefore,  when  replanning  a  route, 
the  participant  and/or  RoboLeader  must  con- 
sider both  the  effectiveness  and  efficiency  of 
the  new  route. 

In  each  scenario,  there  were  six  "events"  that 
required  revisions  to  a  robot's  current  plans  or 
route.  In  some  cases,  robots  had  to  be  rerouted 
to  avoid  certain  areas  (e.g.,  obstructions  and 
hostile  areas);  in  other  cases,  robots  were 
rerouted  to  investigate  nearby  high-priority 
areas  on  the  basis  of  intelligence  messages. 
After  an  event  transpired  (indicated  by  visual 
and/or  auditory  alerts),  the  baseline  participants 
must  notice  that  the  event  had  occurred  and  then 
reroute  the  robot  that  was  affected  by  the  event. 
For  the  RoboLeader  group,  RoboLeader  would 
recommend  plan  revisions  to  the  operator  (by 
presenting  the  new  route  to  the  operator  visu- 
ally on  the  map),  who  could  either  accept  the 
plans  (by  clicking  on  a  button)  or  modify  them 
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TABLE  1:  Experiment  1:  Mean  Operator  Task  Performance  and  Workload  Assessments 

Summary  of  Main 


Baseline 

RoboLeader 

Effects^ 

Measure 

4  Robots 

8  Robots 

4  Robots 

8  Robots 

Targets  (%  detected) 
Situation  awareness 

queries  (%  correct) 
Workload  (NASA  Task 

Load  Index) 

75.0  (10.9) 

46.3  (17.2) 

67.4  (16.3) 

61.3  (13.9) 
26.7  (25.8) 

71.4  (18.1) 

77.0  (14.6) 
48.7  (22.6) 

61.3  (14.9) 

62.7  (18.9) 

23.8  (20.2) 

67.1  (14.7) 

4  robots  >  8  robots 
4  robots  >  8  robots 

8  robots  >  4  robots 

Note.  Standard  deviations  are  presented  in  parentheses. 
^Based  on  statistical  analyses  reported  in  the  Results  section 


as  necessary  (by  dragging  the  waypoints  to  the 
desired  locations).  Out  of  these  six  events,  three 
were  "bottom-up"  (e.g.,  unanticipated  obstacles 
detected  by  the  robots  that  obstructed  their  nav- 
igation; indicated  by  flashing  thumbnails)  and 
three  "top-down"  (e.g.,  intelligence  that  the 
human  operator  received  from  the  intelligence 
network;  indicated  by  auditory  alerts  as  well  as 
appearance  of  icons  on  the  map).  The  partici- 
pants were  told  to  do  their  best  to  perform  both 
the  rerouting  and  the  target  detection  tasks 
instead  of  focusing  on  one  at  the  expense  of  the 
other. 

In  each  scenario,  there  were  five  SA  queries, 
which  were  triggered  on  the  basis  of  time  pro- 
gression (e.g.,  3  min  into  the  scenario).  The  SA 
queries  included  questions  such  as  "Which 
areas  have  the  robots  searched?"  (the  partici- 
pants were  instructed  to  mark  the  searched  areas 
on  a  blank  map),  "Which  of  your  robots  is  the 
closest  to  [area  of  interest]?"  and  so  on.  The 
OCU  screen  was  blank  when  an  SA  query  was 
triggered,  and  only  the  S  A  query  and  the  answer 
box  were  displayed  on  the  screen.  After  each 
scenario,  the  participants  assessed  their  work- 
load (NAS  A-TLX)  and  then  took  a  2-min  break. 
The  entire  experimental  session  lasted  approxi- 
mately 2  hr. 

Dependent  measures  and  data  analysis. 
Dependent  measures  include  number  of  targets 
located  and  identified,  the  operator's  SA  of  the 
mission  environment  as  well  as  awareness  of 
the  status  of  the  individual  robots,  and  per- 
ceived workload.  Mixed-model  ANCOVAs 
were  performed  with  RoboLeader  (with  or 


without  RoboLeader)  as  the  between- subject 
factor  and  number  of  robots  (four  vs.  eight)  as 
the  within- subject  factor.  Participants'  SpA 
(composite  score  of  the  two  spatial  tests)  and 
their  Attentional  Control  survey  scores  were 
used  as  covariates. 

Results 

Target  detection  performance.  Table  1  lists 
several  measures  relating  to  operator  perfor- 
mance and  perceived  workload.  In  terms  of  tar- 
get detection  performance,  the  analysis  revealed 
that  participants  detected  significantly  fewer 
targets  (with  insurgents  and  lEDs  combined) 
when  they  had  eight  robots  compared  with  the 
condition  when  four  robots  were  available,  F(l, 
26)  =  25.35,  p  =  .0001,  ri^p  =  .494.  Participants 
with  higher  SpA  (those  with  higher  composite 
scores  of  spatial  tests)  detected  significantly 
more  targets  than  did  those  with  lower  SpA, 
F(l,  26)  =  8.83,;?  =  .02,  =  .254  (Figure  2). 
The  effects  of  RoboLeader  and  attentional  con- 
trol were  not  statistically  significant. 

SA.  The  analysis  revealed  that  participants' 
SA  was  significantly  poorer  in  the  eight-robot 
condition  compared  with  the  four-robot  condition, 
F(l,  26)  =  13.31,;?  =  .001,  r|^  =  .33  (Figure  3). 
Frequent  video  game  players  had  significantly 
better  SA  than  infrequent  gamers  in  the  Robo- 
Leader condition,  F(l,  11)  =  5.90,  p  =  .02,  ^^  = 
.35,  but  not  in  the  baseline  condition. 

Perceived  workload.  The  analysis  showed 
that  participants  experienced  significantly  higher 
workload  when  there  were  eight  robots 
(M  =  69.3)  versus  four  robots  (M  =  64.3), 
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High  PAC 


Figure  2.  Target  detection  performance  and  effects  of 
operator  spatial  ability  (SpA). 


Basdin^       RoboLeader      Baseline      Ro  ho  Leader 

Figure  4.  Perceived  workload  and  effects  of 
perceived  attentional  control  (PAC). 


Figure  3.  Situation  awareness  queries. 

27)  =  5.24, ;7  =.023,  Ti^p  =  .158  (Figure  4). 
Participants  in  the  RoboLeader  group  assessed 
their  workload  slightly  lower  (M  =  64.1)  than 
did  those  in  the  baseline  group  (M  =  69.4). 
However,  the  difference  failed  to  reach  statisti- 
cal significance.  Participants  with  higher  PAC 
rated  their  workload  as  significantly  lower  than 
did  those  with  lower  PAC;  similarly,  frequent 
video  gamers'  workload  assessments  were  sig- 
nificantly lower  than  those  of  infrequent  gam- 
ers',/? <  .05  (Figure  4). 


Operator's  interaction  with  the  OCU.  Par- 
ticipants' interaction  with  the  OCU  (e.g.,  clicks 
on  the  graphical  control  interface)  was  ana- 
lyzed. Participants'  SpA  was  significantly  cor- 
related with  the  number  of  thumbnail  clicks 
(to  expand  the  video  to  find  targets),  r  =  .448, 
p  =  .006.  Additionally,  participants  in  the  Robo- 
Leader group  spent  significantly  less  time  com- 
pleting their  mission  scenarios  (not  including 
the  time  on  modifying  the  plans)  than  did  those 
in  the  baseline  group  (20.7  vs.  23.8  min),  F(l, 
27)  =  7.12,  p  =  .013,  =  .209.  The  longer 
completion  times  in  the  baseline  condition  sug- 
gest that  the  robots'  routes  were  less  efficient 
than  those  in  the  RoboLeader  condition,  since 
both  groups  were  alerted  about  the  events  that 
required  route  revisions  and  the  time  participant 
spent  on  rerouting  was  excluded  from  the  mis- 
sion completion  times. 

EXPERIMENT  2 

In  Experiment  2,  the  effects  of  unreliable 
RoboLeader  on  operator  performance  were  inves- 
tigated. The  participants'  task,  as  in  Experiment  1, 
was  to  manage  four  robots  with  the  assistance  of 
RoboLeader  while  searching  for  hostile  targets  via 
streaming  video  from  the  robots.  The  reliability  of 
RoboLeader's  solutions  was  manipulated  to  be 
either  FAP  or  MP,  with  a  reliability  level  of  either 
60%  or  90%.  Furthermore,  Experiment  2  simu- 
lated a  multitasking  environment  rather  than  a 
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dual-tasking  environment  as  in  Experiment  1 .  In 
addition  to  the  target  detection  and  route  revision 
tasks,  the  participants  had  to  simultaneously  per- 
form a  gauge  monitoring  task  and  an  auditory 
communication  task.  Finally,  the  visual  density 
(VD)  of  the  simulated  environment  (clutter  of  the 
visual  scene)  was  manipulated;  there  were  twice 
as  many  entities  (virtual  human  characters)  in  the 
high-density  environment  as  in  the  low-density 
environment.  The  experiment  is  a  mixed-model 
design,  with  RoboLeader  imperfection  type  (FAP 
vs.  MP)  and  reliability  level  (60%  vs.  90%)  as  the 
between-subject  factors  and  VD  (high  vs.  low)  of 
the  simulated  environment  as  the  within-subject 
factor. 

Method 

Participants.  For  the  second  experiment,  40 
individuals  (23  males  and  17  females,  mean  age 
23.8  years)  from  the  Orlando,  Florida,  area  par- 
ticipated. They  were  compensated  $  1 5  per  hour 
for  their  time. 

Apparatus.  The  simulator  in  the  first  experi- 
ment was  modified  so  the  reliability  of  Robo- 
Leader was  either  FAP  or  MP,  at  a  level  of  either 
60%  or  90%  (FAP60,  FAP90,  MP60,  and  MP90, 
respectively;  see  Procedure).  The  number  of 


robots  was  reduced  to  four  and  a  gauge  moni- 
toring task  and  a  communication  task  (a  text 
input  area  and  a  submission  button)  were  added 
to  the  user  interface  (Figure  5).  More  details 
about  these  manipulations  and  changes  are  pre- 
sented in  the  Procedure  section. 

The  same  surveys  and  tests  used  in  Experiment 
1  were  employed  in  Experiment  2.  A  modified 
version  of  the  Usability  and  Trust  Questionnaire 
used  in  Chen  and  Terrence  (2009)  assessed  par- 
ticipants' perceived  usability  of  the  RoboLeader 
system  as  well  their  trust  in  the  system.  The 
items  that  measured  participants'  trust  in  the 
system  were  modified  from  the  Trust  Between 
People  and  Automation  questionnaire  (Jian, 
Bisantz,  &  Drury,  2000).  The  questionnaire 
consisted  of  22  questions  on  a  scale  of  1  to  7 
and  included  items  such  as  "The  RoboLeader 
display  can  be  deceptive"  and  "The  RoboLeader 
system  is  dependable." 

Procedure.  Participants  were  randomly 
assigned  to  the  FAP60,  FAP90,  MP60,  or  MP90 
group  (with  10  participants  per  group)  before 
their  sessions  started.  The  procedure  of  the  pre- 
experimental  session  (surveys,  tests,  and  train- 
ing) followed  the  procedure  of  Experiment  1 
and  lasted  approximately  1  hr.  The  type  and 
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reliability  level  of  each  participant's  Robo- 
Leader  condition  (FAP60,  FAP90,  MP60,  or 
MP90)  matched  in  the  training  and  experimen- 
tal scenarios.  The  participants  were  informed 
that  RoboLeader  was  either  FAP  or  MP  and 
"fairly  but  not  always  reliable"  (for  the  90% 
conditions)  or  "not  always  reliable"  (for  the 
60%  conditions). 

The  experimental  session  began  immediately 
after  the  training  session  and  lasted  about  1  hr. 
Each  experimental  session  had  two  scenarios 
(one  with  high  VD  and  one  with  low  density), 
both  lasting  approximately  30  min.  During  the 
scenarios,  participants  used  their  four  robots  to 
locate  10  targets  (insurgents  carrying  weapons) 
while  rerouting  their  robots  around  events  in  the 
remote  environment  (described  later).  In  the 
low-density  scenario,  there  were  about  600 
civilians  throughout  the  scenario,  and  in  the 
high-density  scenario,  there  were  about  600 
civilians  and  600  friendly  soldiers  visible  in  the 
environment.  The  presence  of  friendly  soldiers 
in  the  high-density  scenario  made  the  target 
detection  task  more  difficult,  as  the  friendly  sol- 
diers all  carry  weapons.  The  procedure  of  the 
target  search  task  followed  that  of  Experiment 
1,  and  the  order  of  scenarios  was  counterbal- 
anced across  participants. 

During  the  scenarios,  as  in  Experiment  1, 
several  events  required  revisions  to  the  robots' 
routes.  RoboLeader  and  the  participants  needed 
to  create  new  routes  toward  "high-priority 
areas"  while  avoiding  rerouting  the  robots 
through  "hostile  areas"  or  areas  already  tra- 
versed. As  in  Experiment  1,  participants  were 
told  that  their  objective  was  to  fmish  reconnoi- 
tering  the  area  using  their  robots  in  the  least 
amount  of  time  possible  while  keeping  all  route 
edits  as  close  as  possible  to  the  original  routes. 
In  the  FAP  conditions,  RoboLeader  would  pro- 
vide rerouting  recommendations  that  were  not 
necessary.  Participants  could  check  the  validity 
of  RoboLeader 's  recommendations  by  review- 
ing their  map.  A  true  event  was  associated  with 
an  icon  (a  red  square  for  a  hostile  area  and  a 
blue  square  for  a  high-priority  area;  see  Figure  5), 
but  FAs  were  not. 

In  the  FAP60  scenario,  following  the  signal 
detection  theory  paradigm  (Green  &  Swets, 
1988),  RoboLeader  provided  solutions  to  all 


five  events  that  required  revisions  to  robots' 
routes,  and  it  also  provided  solutions  on  four 
occasions  when  no  events  occurred  and  no  revi- 
sions were  necessary  (i.e.,  five  hits,  four  FAs, 
zero  misses,  and  one  correct  rejection),  making 
a  total  of  10  events,  6  of  which  were  positive.  In 
the  FAP90  scenario,  RoboLeader  provided 
solutions  to  all  5  events  that  required  revisions, 
and  it  also  provided  solutions  on  one  occasion 
when  no  events  occurred.  In  the  MP  scenarios, 
RoboLeader  would  fail  to  provide  solutions 
when  some  events  happened.  In  the  MP60  sce- 
nario, 10  true  events  occurred  that  required 
revisions  to  a  robot's  route,  although  RoboLeader 
provided  solutions  for  only  2  of  them.  In  the 
MP90  scenario,  10  true  events  occurred  and 
RoboLeader  provided  solutions  for  8  of  them. 

In  addition  to  the  tasks  described  already,  the 
participants  simultaneously  performed  a  gauge 
monitoring  task  and  an  auditory  communica- 
tions task.  The  gauge  monitoring  task  (upper 
left  comer  of  the  OCU  under  the  blue 
RoboLeader  message  box)  displayed  four 
gauges  constantly  in  motion  that  entered  an 
upper  or  lower  limit  at  various  prespecified 
times  throughout  the  scenarios.  The  participants 
were  required  to  monitor  the  gauges  and  press  a 
"Reset"  button  when  any  gauge  entered  the 
upper  or  lower  limit  to  put  the  gauges  back  to 
their  normal  levels.  The  auditory  communica- 
tions task  presented  prerecorded  questions  at 
30-s  intervals  during  the  scenarios.  Questions 
included  simple  military-related  reasoning  and 
memory  (i.e.,  call-sign  recognition)  tests. 
Participants  used  a  keyboard  to  enter  their 
responses  for  the  questions  into  the  communi- 
cations panel  on  the  OCU  (adjacent  to  the 
gauges;  see  Figure  5).  As  in  Experiment  1,  the 
participants  were  told  to  do  their  best  to 
perform  all  the  tasks  instead  of  focusing  on 
some  at  the  expense  of  the  others.  Each  sce- 
nario also  contained  five  SA  queries,  which 
followed  the  same  format  and  procedure  as  in 
Experiment  1. 

A  2-min  break  was  given  between  the  experi- 
mental scenarios.  Participants  assessed  their 
workload  using  an  electronic  NASA-TLX 
immediately  after  each  scenario.  Following 
completion  of  both  scenarios,  participants  were 
asked  to  evaluate  the  usability  of  the  RoboLeader 
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system  by  filling  out  the  Usability  and  Trust 
Questionnaire. 

Dependent  measures  and  data  analysis. 
Dependent  measures  include  the  number  of  tar- 
gets located  and  identified,  the  number  of  routes 
edited,  the  operators'  SA  of  the  mission  envi- 
ronment, concurrent  task  performance  (gauge 
monitoring  and  auditory  communications),  and 
perceived  workload.  Mixed-model  ANCOVAs 
were  performed  with  RoboLeader  imperfection 
type  (FAP  vs.  MP)  and  reliability  level  (60% 
[low]  vs.  90%  [high])  as  the  between-subject 
factors  and  VD  (high  vs.  low)  as  the  within- 
subject  factor.  Participants'  composite  SpAtest 
scores  and  their  Attentional  Control  survey 
scores  were  used  as  covariates. 

Results 

Route  editing  task  performance.  Table  2  lists 
several  measures  relating  to  operator  perfor- 
mance and  perceived  workload.  In  terms  of  the 
operator's  route  editing  (i.e.,  automated)  task 
performance,  the  analysis  showed  that  both 
imperfection  type  and  reliability  level  of  Robo- 
Leader significantly  affected  the  percentage  of 
routes  that  participants  edited,  F(l,  35)  =  161.7, 
p  =  .0001,  r|^  =  .82,  andF(l,  35)  =  7.4,;?  =  .01, 
r|^p  =  .18,  respectively  (Figure  6).  Participants 
edited  more  routes  in  the  FAP  condition  and  in 
the  high-reliability  (90%)  condition.  There 
was  a  significant  difference  between  those 
with  higher  SpA  and  lower  SpA,  F(l,  35)  =  7.5, 
p  =  .008,  r|^p  =  .18.  The  difference  between 
the  high-  and  low-SpA  participants  was  espe- 
cially pronounced  in  the  MP60  condition,  with 
high-SpA  participants  editing  74%  of  the  routes 
and  low-SpA  participants  editing  only  56% 
(p  =  .024).  Participants  with  lower  PAC  per- 
formed significantly  worse  than  their  higher- 
PAC  counterparts  in  the  MP  condition,  F(l,  18)  = 
6.0,  p  =  .04,  ri^  =  .25,  but  not  in  the  FAP  condi- 
tion. Further  examination  of  the  data  revealed 
that  the  difference  mainly  came  from  the  low- 
rehability  condition  (73%  vs.  60%),  whereas 
both  groups  performed  at  the  same  level  in  the 
high-reliability  condition  (76%). 

Target  detection  performance.  The  analysis 
revealed  that  there  were  main  effects  of  imper- 
fection type  and  VD,  F(l,  35)  =  45.7,/?  =  .0001, 
^^  =  .57,andF(l,35)  =  12.2,/?  =  .001,r|^  =.26, 


respectively.  Participants  detected  significantly 
fewer  insurgents  in  the  MP  condition  and  in  the 
high-VD  environment.  There  was  a  significant 
difference  between  those  with  higher  SpA  and 
those  with  lower  SpA,  F(l,  35)  =  4.1, p  =  .045, 
r|^p  =  .104  (Figure  7).  Further  analysis  revealed 
that  high- SpA  participants  outperformed  low- 
SpA  participants  to  a  greater  extent  in  the  high- 
VD  condition  than  in  the  low-VD  condition, 
F(l,37)  =  6.09,;7  =  .018,Ti^  =  .141. 

SA.  Participants'  SA  (percentage  of  SA  que- 
ries answered  correctly)  was  significantly  better 
in  the  MP  condition  than  in  the  FAP  condition, 
F(l,  35)  =  S.5,p  =  .003,  ^\  =  .20  (Figure  8). 
Frequent  video  game  players  had  slightly  better 
SA  than  did  infrequent  gamers;  however,  the 
difference  failed  to  reach  statistical  signifi- 
cance, F(l,  38)  =  3.33, p  =  .076. 

Communication  task  performance.  There  were 
no  main  effects  associated  with  the  communica- 
tion task  performance.  There  was  a  significant 
interaction  between  VD  and  participants'  PAC, 
F(l,35)  =  5.4,;?  =  .026,ri^  =  .134. 

Gauge  monitoring  performance.  There  was  a 
significant  interaction  between  VD  and  reliability 
level  of  RoboLeader,  F(l,  35)  =  4.3,  p  =  .047, 
r|^  =  .11.  Higher-PAC  participants  responded 
faster  than  those  with  lower  PAC,  F(l,  35)  = 
84.7,;?  =  .0001,  t\\  =  .31  (Figure  9).  The  differ- 
ence between  high-  and  low-PAC  participants 
was  especially  pronounced  in  low-reliability 
conditions  than  in  high-reliability  conditions. 
Frequent  video  game  players  also  outperformed 
infrequent  gamers,  F(l,  38)  =  8.19,  p  =  .009, 
Tj^  =  .18. 

Individual  differences  in  multitasking  perfor- 
mance. To  further  test  the  effects  of  individual 
differences  in  operators'  overall  multitasking  per- 
formance, multivariate  ANO VAs  were  performed 
on  all  the  task  performance  data.  There  was  a  main 
effect  of  PAC,  F{5,  33)  =  3.60,  p  =  .011,  ^^  =  .35. 
Further  analysis  revealed  that  high-PAC  partici- 
pants outperformed  low-PAC  participants  in  the 
MP  condition,  F(5, 14)  =  4.50,;?  =  .012,  r|^p  =  .62. 
In  the  FAP  condition,  however,  the  difference 
failed  to  reach  statistical  significance.  There  was 
also  a  main  effect  of  participants'  video  gaming 
experience,  F{5,  34)  =  2.42,  p  =  .04,  ^^  =  .26. 
There  failed  to  be  an  overall  effect  of  operator's 
SpA  on  task  performance. 
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Figure  7.  Target  detection  performance  and  effects  of 
operator  spatial  ability  (SpA). 
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Figure  9.  Gauge  monitoring  performance  and  effects 
of  perceived  attentional  control  (PAC). 


Perceived  workload.  The  analysis  showed 
that  both  VD  of  the  target  environment  and  the 
reliability  level  of  RoboLeader  contributed  sig- 
nificantly to  the  participants'  perceived  work- 
load, F(l ,  36)  =  l.l,p  =  .009,  Tj^  =.18,  and F(l , 
36)  =  4.8,  p  =  .036,  r|^  =  .f2,  respectively 
(Figure  10).  Participants  experienced  higher 
workload  in  the  high-VD  condition  as  well  as 
when  the  reliability  level  was  lower. 

Operator's  interaction  with  the  OCU.  Par- 
ticipants made  significantly  more  thumbnail 
clicks  in  the  low-VD  condition  than  in  the 


high-VD  condition,  F(l,  35)  =  6.5,  p  =  .006, 
Tj^  =  .16.  Participants'  SpA  was  also  signifi- 
cantly correlated  with  the  number  of  thumbnail 
clicks,  r  =  33,  p  =  .019.  There  was  no  signifi- 
cant difference  between  higher-  and  lower-PAC 
participants  in  the  aggregate  scores  of  the  trust 
questionnaire. 

GENERAL  DISCUSSION 

We  investigated,  in  two  simulation  experi- 
ments, the  effectiveness  of  human  operators' 
performance  of  military  tasks  when  interacting 
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Figure  10.  Perceived  workload. 


with  an  intelligent  agent,  RoboLeader,  with 
perfect  or  less-than-perfect  reliability.  In  both 
experiments,  participants'  main  task  was  to 
supervise  a  team  of  ground  robots  while  detect- 
ing hostile  targets  via  streaming  video  from  the 
robots.  Whereas  the  first  experiment  involved 
only  the  robotic  route  revision  task  and  the  tar- 
get detection  task,  the  second  experiment  simu- 
lated a  multitasking  environment  in  which  the 
participants  had  to  simultaneously  perform  an 
additional  gauge  monitoring  task  and  an  audi- 
tory communication  task. 

In  the  first  experiment,  although  we  did  not 
fmd  significant  differences  in  target  detection 
between  the  two  groups,  the  participants  in  the 
RoboLeader  group  completed  their  missions  in 
significantly  less  time  than  did  those  in  the 
baseline  group.  On  average,  RoboLeader  saved 
the  participants  approximately  3  min  for  mis- 
sions lasting  20  or  more  minutes  (not  including 
time  on  plan  revisions).  This  suggests  that  the 
RoboLeader 's  plans  were  more  efficient  than 
the  operators'.  The  results  of  Experiment  2 
showed  that  the  type  of  RoboLeader  imperfec- 
tion and  its  reliability  level  both  significantly 
affected  the  participants'  route  editing  perfor- 
mance. As  expected,  participants  edited  more 
routes  in  the  FAP  condition  and  when 
RoboLeader  was  highly  reliable.  Participants' 
spatial  ability  and  attentional  control  both 
played  a  role  in  their  performance  of  route  edit- 
ing, with  the  performance  differences  especially 


pronounced  in  the  MP  and  low-reliability 
conditions. 

In  both  experiments,  one  of  participants' 
tasks  was  to  find  targets  via  streaming  video 
from  the  robots.  In  Experiment  1,  participants 
detected  significantly  fewer  targets  when  there 
were  eight  robots  compared  with  the  four-robot 
condition,  indicating  less  efficiency  with  more 
resources  or  assets.  This  result  is  consistent 
with  the  finding  of  a  recent  study  by  Squire  and 
Parasuraman  (2010),  in  which  the  participants 
performed  worse  (i.e.,  won  fewer  games)  with 
eight  robots  versus  four  robots.  The  results  of 
Experiment  2  showed  that  participants  detected 
fewer  targets  when  the  RoboLeader  was  MP  or 
when  target  VD  was  high.  This  finding  is  con- 
sistent with  that  of  Dixon  and  Wickens  (2006) 
that  concurrent  tasks  (compared  with  automated 
tasks)  are  affected  more  by  MP  systems  than  by 
FAP  systems  because  the  operator  allocates 
more  visual  attention  to  monitor  the  automated 
task  when  the  automation  is  MP.  However,  the 
effectiveness  of  this  elevated  visual  attention  to 
the  automated  task  was  modulated  by  one's 
SpA,  as  we  explain  later. 

From  the  performance  data,  it  does  not 
appear  that  the  difference  in  the  numbers  of 
routes  that  participants  had  to  manually  edit  had 
any  noticeable  effects  on  their  performance  of 
the  concurrent  tasks.  For  example,  for  the  target 
detection  tasks,  there  was  not  a  significant  dif- 
ference between  the  MP60  and  MP90  condi- 
tions, although  participants  were  required  to 
change  eight  routes  on  their  own  in  the  MP60 
condition  and  only  two  routes  in  the  MP90 
condition.  Similarly,  in  Experiment  1,  when 
RoboLeader  was  perfect  and  the  number  of 
events  was  the  same,  there  was  no  difference 
between  RoboLeader  and  baseline-manual  con- 
ditions although  the  manual  conditions  required 
six  edits  and  the  RoboLeader  required  none. 
Taken  together,  these  results  suggest  that  the 
number  of  manual  route  edits  required  was  not 
the  crucial  factor.  It  is,  therefore,  more  likely 
that  the  continuous  visual  monitoring  require- 
ment associated  with  the  MP  conditions  (and 
factors  such  as  VD)  contributed  more  to  the  per- 
formance decrements  observed  for  the  MP  con- 
dition (although  the  MP  group  did  have  better 
SA,  which  is  discussed  later). 
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In  both  experiments,  those  participants  with 
higher  SpA  detected  more  targets  (especially  in 
high-VD  conditions  in  Experiment  2)  and  made 
more  thumbnail  clicks  than  did  those  with  lower 
SpA.  These  results  are  consistent  with  previous 
findings  (Chen,  2010;  Chen  et  al.,  2008;  Chen  & 
Joyner,  2009;  Chen  &  Terrence,  2008, 2009)  that 
individuals  with  higher  SpA  tend  to  exhibit  more 
effective  scanning  performance  and,  therefore, 
are  able  to  detect  more  targets  than  do  those  with 
lower  SpA,  especially  when  visual  processing 
load  is  heavy  (high-VD  conditions  in  Experiment 
2).  The  fact  that  high- SpA  participants  edited 
more  routes  than  did  low-SpA  participants  in 
Experiment  2  also  suggests  more  effective  scan- 
ning by  the  high-SpA  individuals.  In  the  MP60 
condition,  which  required  the  most  visual  moni- 
toring by  the  participants,  high-SpA  participants 
were  able  to  edit  almost  20%  more  of  the  routes 
than  were  their  lower-SpA  counterparts. 

These  findings  support  the  recommendations 
by  Lathan  and  Tracey  (2002)  and  two  recent 
U.S.  Air  Force  studies  (Chappelle,  McMillan,  et 
al,  2010;  Chappelle,  Novy,  et  al,  2010)  that 
military  missions  can  benefit  from  selecting 
personnel  with  higher  SpA  to  operate  robotic 
devices.  Training  interventions  that  could 
enhance  the  spatial  interpretations  required  to 
perform  a  mission  task  might  also  be  of  benefit 
(Rodes,  Brooks,  &  Gugerty,  2005). 

In  terms  of  awareness  of  the  overall  mission 
environments,  participants'  SA  was  signifi- 
cantly better  when  there  were  four  robots  than 
when  there  were  eight  robots  to  keep  track  of 
This  result  is  not  surprising  and  is  consistent 
with  previous  findings  (Crandall,  Goodrich, 
Olsen,  &  Nielsen,  2005;  Squire  &  Parasuraman, 
2010;  Trouvain  &  Wolf,  2002).  Participants'  SA 
was  also  better  in  the  MP  condition  than  in  the 
FAP  condition.  This  suggests  that  the  partici- 
pants scanned  the  map  more  frequently  in  the 
MP  condition  than  in  the  FAP  condition.  Again, 
this  is  consistent  with  Dixon  and  Wickens's 
(2006)  finding  that  MP  systems  drew  operators' 
visual  attention  away  from  the  concurrent 
tasks  to  focus  more  on  the  automated  tasking 
environment.  Interestingly,  participants'  self- 
assessed  trust  in  the  RoboLeader  system  did  not 
differ  significantly  between  the  MP  and  FAP 
groups. 


Although  we  did  not  collect  eye  movement 
data,  we  can  infer  some  of  the  attentional  pro- 
cesses and  strategies  of  the  participants  from 
the  performance  data.  In  their  effort  to  maintain 
optimal  performance  across  the  tasks,  the  par- 
ticipants appeared  to  allocate  more  attentional 
resources  to  the  MP  tasking  environment  than 
to  the  FAP  environment,  as  the  elevated  S  A  per- 
formance for  the  MP  condition  suggested. 
However,  this  biased  attention  allocation  came 
with  a  price — as  the  degraded  target  detection 
performance  in  MP  conditions  indicated. 

There  was  some  evidence  that  frequent  video 
gamers  tended  to  have  somewhat  better  S A  than 
did  infrequent  gamers,  although  the  results  were 
not  always  consistent  (e.g.,  only  in  the  RoboLeader 
condition  but  not  in  the  baseline  condition  in 
Experiment  1 ;  difference  not  reaching  statistical 
significance  in  Experiment  2).  In  another 
RoboLeader  study  employing  different  dynamic 
replanning  tasks,  it  was  observed  that  frequent 
gamers  had  significantly  better  SA  of  the  task- 
ing environment  than  did  infrequent  gamers 
(Chen,  Barnes,  Quinn,  &  Plew,  2011).  Given 
findings  reported  in  Green  and  Bavelier  (2006) 
that  frequent  gamers  tend  to  have  better  visual 
short-term  memory,  it  is  not  surprising  to  find 
them  exhibiting  better  SA  of  the  tasking  envi- 
ronments. Additionally,  Cummings,  Clare,  and 
Hart  (2010)  found  that  frequent  gamers  collabo- 
rated more  with  an  automated  UAS  replanning 
system  (higher  degree  of  consent)  than  did 
infrequent  gamers.  Their  finding  suggests  that 
the  frequent  gamers  in  our  Experiment  1  prob- 
ably worked  better  with  the  RoboLeader  system 
than  did  the  infrequent  gamers,  and  this  more 
effective  collaboration  might  have  contributed 
to  the  frequent  gamers'  better  SA,  especially 
when  RoboLeader  was  reliable. 

Participants  experienced  significantly  higher 
workload  when  there  were  eight  robots  com- 
pared with  the  four-robot  condition,  but  the 
presence  of  RoboLeader  did  not  seem  to  have  a 
significant  effect  on  their  perceived  workload. 
Participants'  workload  assessments  were  also 
higher  when  the  density  of  the  visual  environ- 
ment was  higher  or  when  RoboLeader 's  reli- 
ability level  was  lower. 

Participants'  PAC  was  found  to  have  a  signifi- 
cant effect  on  their  multitasking  performance. 
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especially  the  execution  of  secondary  tasks 
(communication  and  gauge  monitoring).  This 
finding  is  consistent  with  those  of  Chen  and 
Joyner  (2009)  that  participants  performed  at  a 
similar  level  on  the  primary  tasks  (gunnery  and 
robotics),  but  those  with  higher  PAC  performed 
better  on  the  secondary  communication  task  than 
did  those  with  lower  PAC.  These  results  suggest 
that  participants  with  higher  PAC  were  more  able 
to  allocate  their  attentional  resources  effectively 
in  the  multitasking  environment  than  those  with 
lower  PAC,  especially  when  tasking  environ- 
ments became  more  challenging  (e.g.,  low-reli- 
ability conditions  in  Experiment  2).  When  the 
automation  had  low  reliability,  low-PAC  indi- 
viduals did  not  appear  to  be  able  to  allocate  as 
much  attention  to  parts  of  the  tasking  environ- 
ment (e.g.,  gauges)  as  did  high-PAC  individuals 
(Figure  9).  It  was  also  found  that  participants 
with  higher  PAC  consistently  performed  better  in 
the  MP  condition  across  different  tasks  than 
those  with  lower  PAC.  This  is  consistent  with 
Chen  and  Terrence's  (2009)  finding  that  MP 
automated  systems  tended  to  be  more  detrimen- 
tal to  lower  PAC  individuals  than  to  higher  PAC 
individuals. 

An  interesting  difference  between  the  cur- 
rent results  and  those  of  Chen  and  Terrence 
(2009)  was  that  in  the  current  study,  participants 
with  higher  PAC  did  not  exhibit  as  much  under- 
trust  (i.e.,  disuse)  of  the  FAP  system  as  those 
high-PAC  participants  did  in  the  Chen  and 
Terrence  study.  In  the  current  study,  high-PAC 
participants  performed  at  similar  levels  as  low- 
PAC  participants  in  the  FAP  conditions  but  out- 
performed low-PAC  individuals  in  the  MP 
conditions.  The  discrepancy  between  these 
results  and  those  of  Chen  and  Terrence  may  be 
attributable  to  the  different  "costs"  of  scanning 
in  the  two  simulated  environments.  In  the  Chen 
and  Terrence  study,  the  gunner  station  and  the 
robotics  OCU  were  displayed  on  two  separate 
monitors,  whereas  in  the  current  study,  all  tasks 
were  performed  on  the  same  monitor.  Compared 
with  the  current  study,  the  cost  of  scanning  in 
the  Chen  and  Terrence  study  was  greater,  and 
those  of  higher  PAC  clearly  demonstrated 
reduced  compliance  with  the  FAP  automated 
system.  In  the  current  study,  high-PAC  partici- 
pants did  not  show  this  decrement,  likely 


because  of  the  relative  ease  of  verifying  the 
RoboLeader  recommendations  on  the  map  by 
checking  the  icons. 

It  is  interesting  to  note  that  while  it  was  con- 
siderably easier  to  verify  the  validity  of  the  alerts 
in  the  current  study,  participants  with  low  PAC 
performed  more  poorly  in  the  MP  conditions  did 
than  those  with  high  PAC,  just  as  the  results  of 
the  Chen  and  Terrence  (2009)  study  showed.  A 
likely  reason  for  this  phenomenon  is  that  MP  sce- 
narios required  continuous  scanning  of  the  map 
to  find  new  icons.  This  requirement  made  the 
task  similar  to  a  "change  detection"  task, 
although  performed  in  a  multitasking  environ- 
ment. The  current  results  suggest  that  low-PAC 
individuals  cannot  detect  changes  as  effectively 
as  their  high-PAC  counterparts,  likely  because  of 
their  poorer  attentional  management  abilities. 
The  way  the  low-PAC  participants  interacted 
with  the  automated  system  in  the  current  experi- 
ment was  consistent  with  the  "cognitive  miser" 
phenomenon  described  in  Feldman  Barrett  et  al. 
(2004).  The  phenomenon  states  that  low-PAC 
individuals,  because  of  their  limited  attentional 
resources,  tend  to  reduce  their  information  pro- 
cessing demands  by  simplifying  their  task(s) 
(e.g.,  relying  on  automation  to  help  them  with 
their  tasks).  Depending  on  the  context,  this  over- 
simphfication  (e.g.,  overreliance  on  automation) 
may  have  very  undesirable  consequences  (e.g., 
MP  condition)  when  the  aids  fail  to  provide 
anticipated  assistance. 

On  the  other  hand,  notably,  there  were  no 
observable  trade-off  strategies  adopted  by  the 
low-PAC  participants  (their  performance  across 
the  different  tasks  was  uniformly  low  in  the  MP 
condition).  In  other  words,  the  low-PAC  partici- 
pants were  not  found  to  conserve  cognitive 
resources  on  one  task  to  perform  better  on  the 
other  tasks.  This  performance  decrement,  there- 
fore, seems  to  reflect  something  more  related  to 
attentional  abilities  than  to  deliberate  strategies. 
Future  research  should  investigate  factors  con- 
tributing to  this  deficiency  by  low-PAC  indi- 
viduals (e.g.,  working  memory  capacity  and 
cognitive  flexibility;  Buhner,  Konig,  Pick,  & 
Krumm,  2006;  Feldman  Barrett  et  al,  2004; 
Youmans,  Figueroa,  &  Kramarova,  2011)  and 
possible  training  or  interfaces  design  strategies 
to  mitigate  them. 
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Participants'  video  gaming  experience  had  a 
significant  impact  on  their  overall  multitasking 
performance.  These  results  are  consistent  with 
previous  findings  (Green  &  Bavelier,  2006) 
suggesting  that  video  game  play  is  associated 
with  greater  visual  short-term  memory  and 
faster  information  processing,  which  in  turn 
may  have  contributed  to  game-playing  partici- 
pants' superior  multitasking  performance  in  the 
current  study.  These  results  are  consistent  with 
the  findings  of  one  recent  U.S.  Air  Force  study 
(McKinley,  Mclntire,  &  Funke,  2010)  that  fre- 
quent video  gamers  outperformed  infrequent 
gamers  on  robotics  (UAS)  tasks  and,  in  some 
cases,  performed  as  well  as  experienced  pilots. 
These  results  also  support  the  conclusion  of  an 
U.S.  Air  Force  study  (Triplett,  2008)  based  on 
interviews  of  UAS  pilots  that  gamers'  superior 
visual  information-processing  skills  may  be 
able  to  translate  into  superior  robotics  manage- 
ment performance. 

CONCLUSION 

In  the  current  study,  we  investigated  the 
effects  of  an  intelligent  agent,  RoboLeader,  on 
human  operators'  performance  of  supervising 
multiple  robots  to  complete  military  recon- 
naissance missions  in  a  dual-task  and  a  multi- 
tasking environment.  Overall,  it  appears  that 
RoboLeader,  when  perfectly  reliable,  was 
effective  in  reducing  the  operators'  mission 
times  in  target  search  tasks,  although  signifi- 
cant benefits  of  RoboLeader  on  the  operators' 
concurrent  task  performance  and  workload 
were  not  observed. 

Results  of  Experiment  2  show  that  the  type  of 
RoboLeader  imperfection  affected  operator's 
performance  of  tasks  involving  visual  scanning 
(target  detection,  route  editing,  and  SA). 
Furthermore,  there  was  a  consistent  effect  of  VD 
for  multiple  performance  measures.  Participants' 
self-assessed  attentional  control  and  video  gam- 
ing experience  was  found  to  affect  their  overall 
multitasking  performance.  Across  experiments, 
participants  with  higher  SpA  consistently  outper- 
formed those  with  lower  SpA  in  tasks  that 
required  the  most  visual  scanning  (e.g.,  target 
detection  and  thumbnail  clicks),  regardless  of  the 
experimental  conditions.  Future  research  should 
investigate  training  interventions  (e.g.,  attention 


management)  and/or  user  interface  designs  (e.g., 
multimodal  cueing  displays)  that  can  mitigate 
performance  shortfalls  of  those  with  lower  SpA 
and  attentional  control  (Chen,  Barnes,  &  Harper- 
Sciarini,  201 1 ;  Chen,  Haas,  &  Barnes,  2007;  Dux 
et  al.,  2009). 
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KEY  POINTS 

•  Intelligent  agents,  such  as  RoboLeader,  can  bene- 
fit the  overall  human-robot  teaming  performance. 
However,  the  effects  of  unreliable  automation, 
tasking  requirements,  and  individual  differences 
are  complex,  and  their  interactions  with  one 
another  have  different  ways  to  affect  human-auto- 
mation interaction  in  multitasking  environments. 

•  The  type  of  RoboLeader  imperfection  (false 
alarm  vs.  miss  prone)  affected  operator's  perfor- 
mance of  tasks  involving  visual  scanning  (target 
detection,  route  editing,  and  situation  awareness). 
The  effectiveness  of  participants'  elevated  visual 
attention  to  the  miss-prone  environment  was 
modulated  by  individual  differences  in  spatial 
ability  and  attentional  control. 

•  Participants'  attentional  control  and  video  gam- 
ing experience  affected  their  overall  multitask- 
ing performance.  The  positive  effects  of  higher 
attentional  control  were  most  evident  when  the 
automated  system  (RoboLeader)  was  miss  prone. 

•  In  both  experiments,  participants  with  higher 
spatial  ability  consistently  outperformed  their 
low-spatial-ability  counterparts  in  tasks  that 
required  effective  visual  scanning:  They  scanned 
the  environment  faster  and  detected  more  targets. 
The  performance  differences  were  especially  pro- 
nounced when  visual  processing  load  was  heavy 
(high  visual  density  and  low  system  reliability). 
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A  military  targeting  environment  was  simulated  to  examine  the  effects  of  an  intelligent  route-planning  agent 
RoboLeader,  which  could  support  dynamic  robot  re-tasking  based  on  battlefield  developments,  on  the  performance 
of  robotics  operators.  We  manipulated  the  level  of  assistance  (LOAs)  provided  by  RoboLeader  as  well  as  the 
presence  of  a  visuaHsation  tool  that  provided  feedback  to  the  participants  on  their  primary  task  (target 
encapsulation)  performance.  Results  showed  that  the  participants'  primary  task  benefited  from  RoboLeader  on  all 
LOAs  conditions  compared  to  manual  performance;  however,  visuaHsation  had  Httle  effect.  Frequent  video  gamers 
demonstrated  significantly  better  situation  awareness  of  the  mission  environment  than  did  infrequent  gamers.  Those 
participants  with  higher  spatial  abiUty  performed  better  on  a  secondary  target  detection  task  than  did  those  with 
lower  spatial  abiUty.  Finally,  participants'  workload  assessments  were  significantly  lower  when  they  were  assisted  by 
RoboLeader  than  when  they  performed  the  target  entrapment  task  manually. 

Practitioner  Summary:  This  study  demonstrated  the  utility  of  an  intelligent  agent  for  enhancing  robotics  operators' 
supervisory  control  performance  as  well  as  reducing  their  workload  during  a  complex  urban  scenario  involving 
moving  targets.  The  results  furthered  the  understanding  of  the  interplay  among  level-of-autonomy,  multitasking 
performance  and  individual  differences  in  military  tasking  environments. 

Keywords;  human-robot  interaction;  supervisory  control;  level  of  autonomy;  intelligent  agent;  military;  individual 
differences 


1.  Introduction 

Robots  are  becoming  an  essential  part  of  the  battlefield  (Chen  et  al.  2011a).  Thousands  of  robots  are  currently 
working  side-by-side  with  soldiers,  and  the  tasks  they  are  used  for  are  evolving  in  complexity  -  from  load-carrying, 
casualty  extraction,  hazardous  material  detection,  counter  explosive  devices,  building  mapping/clearing,  to 
firefighting  (Purdy  2008,  Greenemeier  2010,  Osborn  2011).  In  the  future  battlefield.  Soldiers  may  be  given  multiple 
tasks  to  perform  concurrently,  such  as  navigating  a  robot  while  conducting  surveillance,  maintaining  local  security 
and  situation  awareness  (SA),  and  communicating  with  fellow  team  members.  The  possibihty  of  a  robotic  battlefield 
creates  a  number  of  human  factors  as  well  as  ethical  issues  related  to  non-human  intelhgence  conducting  combat 
missions  (Barnes  and  Evans  2010,  Singer  2010).  One  obvious  issue  is  that  the  prohferation  of  intelHgent  systems 
could  easily  overwhelm  the  human  operators'  ability  to  adequately  supervise  these  systems.  Hundreds  of  robots, 
both  aerial  and  ground,  may  share  the  battlefield  with  hundreds  of  manned  systems  and  conduct  numerous  missions 
concurrently.  In  these  situations,  the  military  may  not  be  able  to  afford  the  manpower  to  have  operators  control 
individual  systems;  instead,  future  missions  will  likely  require  single  operators  to  supervise  multiple  systems 
simultaneously  (Cummings  et  al.  2010). 

Past  research  has  shown  that  human  operators  are  often  unable  to  control  multiple  robots/agents 
simultaneously  in  an  effective  and  efficient  manner  (Schurr  2007,  Chen  et  al.  2008).  As  the  size  of  the  robot  team 
increases,  human  operators  may  fail  to  maintain  adequate  SA  when  their  attention  is  constantly  switching  between 
the  robots.  Cognitive  resources  may  also  be  overwhelmed  by  the  numerous  intervention  requests  from  the  robots 
(Wang  et  al.  2009a,  Lewis  et  al.  2010).  Wang  et  al.  (2009a)  reviewed  a  number  of  studies  on  supervisory  control  of 
multiple  ground  robots  for  target  detection  tasks  and  concluded  that  in  order  to  be  effective,  'the  Fan-out  plateau 
hes  somewhere  between  4  and  9+  robots  depending  on  the  level  of  robot  autonomy  and  environmental  demands' 
(p.  143)  -  limits  very  close  to  the  classical  (seven  plus  or  minus  two)  findings  for  span  of  apprehension  (Miller  1956). 
Unfortunately,  increases  in  autonomy  often  present  its  own  set  of  problems,  including  tunnel  vision,  misuse  and 
disuse  of  automated  systems,  complacency,  and  loss  of  SA  (Parasuraman  and  Riley  1997,  Chen  et  al.  2011a).  For 
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a  detailed  review  on  human  performance  issues  supervisory  control  of  multiple  robots  and  potential  user  interface 
designs,  see  Cummings  et  al.  (2010)  and  Chen  et  al.  (2011a). 

7.7.    Systems  to  enhance  human  supervisory  control  of  multiple  robots 

Researchers  have  proposed  a  number  of  solutions  to  the  potential  issues  of  a  robotic  battlefield,  such  as  setting  up  a 
robot  call  center  in  which  robots  can  query  the  human  operator  only  when  there  is  a  problem  (Wang  et  al.  2009b). 
The  operator  can  make  the  necessary  adjustments  but  are  not  required  to  monitor  robots  continuously.  A  potential 
problem  with  this  solution  is  that  it  assumes  that  the  robot  can  rehably  self-diagnose  its  own  problems;  additionally, 
the  number  of  operator-robot  interactions  is  expected  to  increase  exponentially  during  the  heat  of  combat,  making 
the  call  center  ineffective  during  the  most  critical  time  periods.  As  a  potential  safeguard,  a  number  of  researchers 
have  suggested  algorithms  that  share  control  responsibihty  among  robots  and  humans  as  a  function  of  either  the 
robots'  behaviour  or  the  operator's  cognitive  state  (Parasuraman  et  al.  2007,  Miller  and  Parasuraman  2007, 
Cosenzo  et  al.  2010,  Goodrich  2010).  Closely  aHgned  concepts  involve  play-book  solutions  that  permit  the  operator 
to  insert  pre-programmed  algorithmic  solutions  that  control  robots  during  difficult  mission  segments  (Miller  and 
Parasuraman  2007,  Fern  and  Shively  2009).  This  generic  class  of  adaptive  systems  is  designed  to  keep  operators  in 
the  decision  loop  while  keeping  the  overall  supervisory  burden  within  manageable  cognitive  limits.  However,  while 
this  approach  mitigates  problems  during  high  workload  segments,  it  does  not  overcome  cognitive  limitations  when 
the  number  of  human-robot  interactions  surpasses  human  cognitive  capacity  (Lewis  et  al.  2010). 

Research  shows  that  autonomous  cooperation  between  robots  can  aid  the  performance  of  the  human  operators 
(Lewis  et  al.  2010)  and  enhance  the  overall  human-robot  team  performance  (Schurr  2007).  Human  operators' 
involvement  in  mixed-initiative  teams  will  still  be  required  for  the  foreseeable  future,  however,  especially  in  situations 
involving  critical  decision  making.  Human  operators'  decision  making  may  be  influenced  by  'implicit  goals'  that  the 
robots  are  not  aware  of  (i.e.  are  not  programmed  into  the  behaviors  of  the  robots;  Linegang  et  al.  2006)  and  real-time 
developments  on  the  battlefield  that  may  require  the  human  operator  to  change  plans  for  individual  robots  or  the 
entire  robotic  team.  Effective  communication  between  the  human  operator  and  robots  then  becomes  critical  in 
ensuring  mission  success.  Past  research  has  demonstrated  the  effectiveness  of  a  robot  proxy  to  enhance  shared 
understanding  between  the  human  operator  and  the  robot  in  an  exploration  task  (Stubbs  et  al.  2008).  The 
communication  mechanism  was  based  on  a  common  ground  collaboration  model  and  improved  the  human  operator 
performance  by  assisting  in  the  creation  of  more  accurate  plans,  more  efficient  planning  (fewer  planning  repetitions), 
faster  task  performance,  as  well  as  a  better  mental  model  of  the  capabilities  of  the  robot  (Stubbs  et  al.  2008). 

7.2.  RoboLeader 

To  achieve  a  better  balance  of  enhancing  autonomy  and  capability  while  simpHfying  human-robot  interaction,  an 
inteUigent  agent  called  RoboLeader,  a  robotic  surrogate  that  could  help  the  human  operator  coordinate  a  team  of 
ground  robots,  was  developed  under  the  US  Army  Research  Laboratory's  (ARL)  Director's  Research  Initiative 
Program  (Chen  et  al.  2011b,  Chen  and  Barnes  2012).  In  other  words,  instead  of  directly  managing  the  robot  team, 
the  human  operator  only  dealt  with  RoboLeader;  consequently,  the  operator  could  better  focus  on  the  other  tasks 
requiring  attention.  In  Chen  and  Barnes  (2012),  RoboLeader  had  the  capabiHties  of  revising  route  plans  for  the 
robots  based  on  static  battlefield  inteUigence  and  developments  (e.g.  potential  attacks,  road  blocks,  high  priority 
areas,  etc).  Chen  and  Barnes  investigated  the  effectiveness  of  RoboLeader  in  a  human-in-the-loop  simulation 
experiment  (in  which  30  individuals  participated)  and  compared  the  operators'  target  detection  performance  in  a 
four-  and  eight-robot  condition.  The  results  showed  that  participants  detected  significantly  fewer  targets  with  eight 
robots  versus  four  robots.  Although  there  were  no  significant  differences  between  the  RoboLeader  and  baseHne  (no 
RoboLeader)  conditions  for  target  detection,  the  RoboLeader  group  reduced  their  mission  completion  times  by 
approximately  13%  compared  to  the  basehne  group.  Additionally,  those  participants  with  higher  spatial  abiHty 
detected  more  targets  than  did  those  with  lower  spatial  abiHty.  Participants  experienced  significantly  higher 
workload  with  eight  robots  compared  to  the  four-robot  condition,  and  those  with  better  attentional  control 
reported  lower  workload  than  did  those  with  poorer  attentional  control. 

7.5.    Current  study 

The  purpose  of  the  current  study  was  to  investigate  the  efficacy  of  agent/human  cooperation  during  more 
challenging  mission  tasking  while  varying  the  level  of  operator  interaction  with  RoboLeader.  In  the  current  study. 
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the  capabilities  of  RoboLeader  were  expanded  to  deal  more  specifically  with  dynamic  re-tasking  requirements  for 
persistent  surveillance  of  a  simulated  urban  environment  based  on  various  battlefield  developments,  as  well  as 
coordination  between  unmanned  aerial  vehicles  (UAVs)  and  unmanned  ground  vehicles  (UGVs)  in  pursuit  of 
moving  targets  in  urban  environments  (for  more  information  on  the  RoboLeader  algorithm,  see  Chen  et  al.  2011b). 
More  specifically,  the  participants  used  four  UGVs  to  'entrap'  a  primary  moving  target  (a  truck)  via  a  map  that 
displays  the  location  of  the  target  via  a  UAV  Hnk  while  monitoring  the  streaming  videos  from  the  four  UGVs  to 
look  for  other  secondary  targets  (insurgents)  in  the  remote  environments.  This  dynamic  tasking  environment  is 
significantly  more  challenging  than  what  was  investigated  in  the  past  RoboLeader  studies  and  it  captures  the 
complexity  inherent  in  future  military  environments  requiring  robot  re-routing  decisions  in  reference  to  mobile 
targets  while  optimising  detection  of  targets  of  opportunity. 

RoboLeader's  role  was  to  coordinate  multiple  robots  in  concert  with  the  human  decision  maker.  We 
manipulated  the  level  of  assistance  (LOAs)  provided  by  RoboLeader  and  examined  its  effect  on  the  operator's 
performance  (i.e.  primary  target  entrapment,  percentage  of  secondary  targets  detected  and  SA  of  the  mission 
environment)  and  workload.  The  four  levels  of  manipulation  were  as  follows:  Manual  (without  Visuahsation), 
Semi-Autonomous  without  Visualisation  {Semi-Auto  w/o  Vis),  Semi-Autonomous  with  Visualisation  (Semi-Auto 
w/  Vis)  and  Fully  Automated  (Auto;  with  Visualisation;  see  Procedure).  The  Semi- Autonomous  condition  was 
divided  into  two  conditions  so  that  the  effect  of  the  visuahsation  tool  could  be  isolated  and  evaluated.  The 
visualisation  tool  informed  the  participant  of  the  synchronisation  of  the  robots  as  well  as  overall  target 
entrapment  effectiveness  based  on  the  movement  of  the  target  (see  Apparatus).  The  four  conditions,  therefore, 
represented  four  levels  of  assistance  of  RoboLeader  (Figure  1),  each  adding  more  capability  to  the  previous  level  - 
(1)  manual  (baseline);  (2)  the  waypoint  completion  capability  added  to  the  Semi-Auto  w/o  Vis;  (3)  the 
visualisation  tool  added  to  the  Semi-Auto  w/Vis;  (4)  the  automated  encapsulation  and  route  endpoint  planning 
capabihty  added  to  the  Auto  condition.  In  terms  of  level  of  automation,  RoboLeader's  route-planning  capability 
resembles  the  fifth  level  ('execute  that  suggestion  if  the  human  approves')  described  in  Parasuraman  et  al.  (2000,  p. 
287).  In  the  previous  RoboLeader  studies,  RoboLeader  was  either  fully  automated  (100%  rehable)  or  imperfectly 
automated  (miss  prone  or  false-alarm  prone);  the  tasking  environments  were  also  considerably  more  static  than 
the  current  one  (Chen  and  Barnes  2012).  Thus  we  expanded  on  our  previous  research  not  only  by  increasing  the 
complexity  level  (i.e.  dynamic  re-tasking)  but  also  by  varying  the  LOAs  available  to  the  human  operators  for 
human-agent  interactions. 

Finally,  we  investigated  the  effects  of  individual  differences  on  the  operators'  robotics  control  as  well  as 
multitasking  performance.  Specifically,  we  investigated  the  effects  of  participants'  spatial  abihty,  attentional  control 
and  gaming  experience  (using  the  same  tests  and  surveys  as  in  Chen  and  Barnes  2012).  The  rest  of  this  section  briefly 
reviews  the  individual  differences  factors  investigated  in  the  current  study. 


L4,    Individual  differences  in  spatial  ability,  attentional  control  and  gaming  experience 

In  the  current  study,  we  sought  to  evaluate  whether  individual  differences  in  spatial  ability,  attentional  control,  and 
video  gaming  experience  might  impact  the  operator's  performance.  Our  simulated  tasking  environments  required 
the  participants  to  process  a  large  amount  of  spatial  information  from  multiple  sources  (even  when  working  with  the 
RoboLeader  agent).  Past  research  suggests  individual  differences  in  spatial  abihty,  attentional  control  and  video 
gaming  experience  can  play  significant  roles  in  such  tasking  environments  where  effective  scanning  and  multitasking 
are  critical  to  performance  success.  The  current  study  sought  to  evaluate  the  interaction  between  these  individual 
differences  factors  and  the  LOAs  provided  by  RoboLeader.  To  date,  there  has  been  minimal  research  on  human- 
agent  interaction  investigating  all  these  factors  in  one  single  study.  The  rest  of  the  section  briefly  reviewed  each 
factor  and  its  relevance  to  the  current  study. 

Spatial  ability  has  been  found  to  be  a  significant  factor  in  certain  visual  display  domains  (Stanney  and  Salvendy 
1995),  virtual  environment  navigation  (Chen  et  al.  2000),  target  search  task  (Chen  et  al.  2008,  Chen  and  Joyner  2009, 
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Figure  1.    Four  levels  of  assistance  of  RoboLeader  (WP  =  waypoint;  EP  =  endpoint). 


163 


Chen  and  Terrence  2008, 2009,  Chen  2010,  Chen  and  Barnes  2012)  and  robotics  task  performance  (Lathan  and  Tracey 
2002,  Cassenti  et  al.  2009,  Baber  et  al.  2011).  For  example,  Stanney  and  Salvendy  (1995)  found  that  high  spatial 
individuals  outperformed  those  with  low  spatial  abihty  on  tasks  that  required  visuo-spatial  representations  to  be 
mentally  constructed  (i.e.  mental  model  of  hierarchical  menu  systems).  Previous  studies  also  found  that  spatial  abihty 
was  correlated  with  robot  teleoperation  tasks  such  as  robot  navigation  (Lathan  and  Tracey  2002,  Cassenti  et  al.  2009). 
Lathan  and  Tracey  (2002)  showed  that  people  with  higher  spatial  abihty  (determined  based  on  a  test  battery  of  two 
spatial  recognition  and  two  spatial  manipulation  tests)  performed  better  in  a  robot  teleoperation  task  through  a  maze 
in  terms  of  both  speed  and  accuracy.  A  recent  study  by  Baber  et  al.  (2011)  showed  that  low  spatial  participants 
responded  significantly  more  slowly  when  controUing  multiple  robots.  Our  previous  research  showed  that  individuals 
with  higher  spatial  ability  exhibited  more  effective  visual  scanning  and  target  detection  performance  (Chen  et  al.  2008, 
Chen  and  Joyner  2009,  Chen  and  Terrence  2008,  2009,  Chen  2010,  Chen  and  Barnes  2012).  Chappelle  et  al.  2010a, 
2010b  interviewed  53  subject  matter  experts  about  abihties  that  were  critical  to  effective  performance  of  UAV  control 
tasks  in  terms  of  piloting  and  sensor  operations  -  spatial  ability  was  identified  as  an  important  factor  for  both  tasks. 
Based  on  these  previous  findings,  therefore,  we  expected  participants  with  higher  spatial  ability  would  outperform 
their  lower  spatial  counterparts  in  tasks  that  required  effective  visual  scanning  and  spatial  reasoning. 

In  addition  to  spatial  ability,  the  relationship  between  attentional  control  and  multitasking  performance  was 
also  examined.  Attentional  control  is  defined  as  one's  ability  to  focus  and  shift  attention  in  a  flexible  manner  (e.g. 
ability  to  re-direct  attention  to  certain  aspect  of  the  environment  without  being  distracted  by  other  elements  of  the 
environment;  Derryberry  and  Reed  2002).  Several  studies  have  shown  that  there  are  individual  differences  in 
multitasking  performance,  and  some  people  are  less  prone  to  performance  degradation  during  multitasking 
conditions  (Rubinstein  et  al.  2001,  Schumacher  et  al.  2001).  There  is  some  evidence  that  attention- switching 
flexibihty  can  predict  performance  of  such  diverse  tasks  as  flight  training  and  bus  driving  (Kahneman  et  al.  1973). 
There  is  also  evidence  that  people  with  better  attention  control  can  allocate  their  attention  more  flexibly  and 
effectively  (Derryberry  and  Reed  2002),  and  the  abihty  to  allocate  attention  flexibly  is  related  to  one's  working 
memory  capacity  (Bleckley  et  al.  2003,  Feldman  Barrett  et  al.  2004).  According  to  a  recent  US  Air  Force's  survey  of 
subject  matter  experts  on  the  performance  of  operators  of  UAVs  (Chapelle  et  al.  2010a),  attentional  control  is  one 
of  the  most  important  abilities  that  would  affect  an  operator's  performance  since  the  robotics  control  task  is 
inherently  multitasking  (e.g.  sensor  manipulation,  tracking,  communication,  etc.).  According  to  Feldman  Barrett 
et  al.  (2004),  those  with  lower  attentional  control  tend  to  take  the  'cognitive  miser'  approach  when  deahng  with 
complex  information  processing  in  order  to  reduce  the  attentional  control  requirements.  When  dealing  with 
automation,  therefore,  it  is  hkely  that  operators  with  different  levels  of  attention  switching  abihties  may  react 
differently  to  automated  systems.  In  other  words,  operators'  behaviours  of  comphance  with,  and  rehance  on, 
automation  may  be  altered  based  on  their  abihty  to  effectively  switch  their  attention  among  the  systems.  For 
example,  the  automation-induced  complacency  effect  repeatedly  demonstrated  in  previous  research  (Dzindolet  et  al. 
2001,  Thomas  and  Wickens  2004,  Young  and  Stanton  2007,  Parasuraman  and  Manzey  2010)  may  be  more  severe 
for  poor  attentional  control  individuals  compared  with  those  with  better  attentional  control.  This  phenomenon  has 
been  demonstrated  in  Chen  and  Terrence  (2009)  and  Chen  and  Barnes  (2012)  and  was  further  tested  in  the  current 
experiment,  which  employed  a  simulation  environment  considerably  different  from  those  of  the  previous 
experiments. 

Finally,  the  current  study  sought  to  examine  the  relationship  between  participants'  video  gaming  experience  and 
their  task  performance  as  well  as  SA  of  the  mission  environment.  Chen  and  Barnes  (2012)  demonstrated  that 
frequent  gamers  (i.e.  those  who  play  video  games  daily  or  weekly)  could  multitask  better  and  had  better  SA  than  did 
infrequent  gamers  (i.e.  those  who  play  monthly  or  less  frequently  than  monthly).  According  to  Green  and  Bavelier 
(2006)  and  Hubert-WaUander  et  al.  (2010),  experienced  action  video  game  players,  compared  to  infrequent/non 
gamer,  were  found  to  perform  significantly  better  on  tasks  that  required  visuo-spatial  selective  attention,  multiple 
object  tracking,  fast  process  of  visual  information  and  imagery,  and  flexibihty  in  attention  allocation.  Therefore,  we 
expected  frequent  gamers  would  outperform  infrequent  gamers  in  our  visually  demanding  task  environment  in 
terms  of  target/event  detection  and  monitoring  of  the  mission  environment. 


2.  Method 

2.7.  Participants 

Twenty-eight  individuals  [25  males  and  3  females;  mean  age  =  21.4  (SD  =  4.3),  min=18,  max  =  34]  from  the 
Orlando,  FL  area  participated  in  the  experiment.  Out  of  the  28  participants,  none  had  any  prior  mihtary 
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experiences.  The  majority  of  the  participants  were  students  from  a  university  in  the  Orlando  area,  and  six  had 
completed  four  years  of  college.  Overall,  the  participants  were  similar  in  age  and  in  background  to  potential  mihtary 
operators  before  training.  Based  on  their  self-reported  gaming  experience,  20  of  them  played  video  games  either 
daily  or  weekly.  These  individuals  were  classified  as  frequent  gamers.  The  other  eight  played  video  games  either 
monthly  or  with  even  less  frequency.  These  individuals  were  classified  as  infrequent  gamers.  Participants  received 
$15/hour  for  their  time. 


2.2.  Apparatus 

2.2.1.  Simulator 

A  modified  version  of  the  Mixed  Initiative  Experimental  (MIX)  Testbed  was  used  as  the  simulator  for  this 
experiment.  The  MIX  Testbed  is  a  distributed  simulation  environment  for  investigation  into  how  robots  are  used 
and  how  automation  affects  human  operator  performance  (Barber  et  al.  2008).  The  Operator  Control  Unit  (OCU) 
of  the  MIX  Testbed  (Figure  2a)  was  modelled  after  the  Tactical  Control  Unit  developed  under  the  ARL  Robotics 
Collaborative  Technology  Alliance.  This  platform  includes  a  camera  payload  and  supports  multiple  levels  of 
automation.  Users  can  send  mission  plans  or  teleoperate  the  platform  with  a  computer  mouse  while  being  provided 
video  feeds  from  the  camera  payload.  Typical  tasks  include  reconnaissance  and  surveillance.  The  RoboLeader 
software  was  integrated  in  the  MIX  OCU  and  had  the  capabiHty  of  collecting  information  from  subordinate  robots 
with  limited  autonomy  (e.g.  collision  avoidance  and  self-guidance  to  reach  target  locations),  making  tactical 
decisions  and  coordinating  the  robots  by  issuing  commands,  waypoints,  or  motion  trajectories  (see  Chen  et  al. 
2011b  for  a  detailed  description  of  the  algorithm  of  RoboLeader;  Figure  2a).  Participants  entered/modified 
waypoints  or  endpoints  for  robots'  routes  by  selecting  the  appropriate  button  at  the  top  of  the  OCU  and  then  chck 
on  the  spot  of  the  map  where  the  point  should  be  placed  (see  Figure  2b) 


2.2.2.  RoboLeader 

The  RoboLeader  user  interface  is  shown  in  Figure  2a.  The  map  was  located  in  the  top  left  portion  of  the  display 
and  showed  the  location  of  the  moving  target  (a  slowly  moving  truck)  via  a  link  to  a  UAV  flying  over  the  mission 
area  (i.e.  the  icon  representing  the  truck  was  displayed  as  a  red  arrow  to  show  the  direction  of  movement  and 
continuously  moved  on  the  map  display  based  on  the  truck's  actual  movement,  which  was  updated  from  the  UAV). 
The  right  half  of  the  screen  was  divided  into  four  separate  camera  feeds,  streaming  from  the  four  UGVs.  The 
visuahsation  display  (one  horizontal  bar  and  four  vertical  bars)  was  located  in  the  lower  left  corner  of  the  OCU  (for 
design  principles  for  information  visuahsation,  see  Robertson  et  al.  2009).  The  purpose  of  the  visuahsation  display 
was  to  provide  real-time  performance  feedback  to  the  participants  regarding  their  target  encapsulation  task  and  the 
scoring  algorithm  took  into  account  of  the  moving  target's  heading  as  well  as  the  absolute  distances  between  the 
target  and  the  UGV  (Chen  et  al.  201  lb).  The  four  vertical  bars  had  the  words  'NEAR'  and  'AWAY'  at  the  top  and 
bottom  of  each  bar,  respectively,  and  represented  the  scores  for  each  robot's  progress  in  getting  closer  to  the 
moving  target;  the  horizontal  bar  represented  the  aggregate  entrapment  score  of  the  robots'  plans  (in  terms  of 
blockage  of  the  target's  escape  routes).  As  the  target  moved,  RoboLeader  calculated  the  scores  based  on  each 
robot's  plan  as  well  as  the  speed  and  movement  of  the  target  and  showed  the  scores  via  the  bar  graphs.  As  a  UGV 
approached  the  target,  its  associated  green  vertical  bar  would  increase  in  size  and  move  closer  to  the  'NEAR'  end 
(longer  green  bars  meaning  closer  distances);  conversely,  as  a  UGV  moved  further  away  from  the  target,  its 
associated  red  vertical  bar  would  increase  in  size  and  move  closer  to  the  'AWAY'  end  (longer  red  bars  meaning 
farther  distances).  Additionally,  a  red  arrow  was  displayed  to  the  right  of  each  graph  if  a  UGV  was  moving  away 
from  the  target  (see  Figure  2a).  The  horizontal  bar,  on  the  other  hand,  indicated  whether  the  target's  escape  routes 
were  properly  blocked  by  at  least  two  UGVs  based  on  the  current  plans  (longer  bar  meaning  better  blockage).  The 
score  could  be  perfect  if  the  target's  escape  routes  were  properly  blocked  by  two  robots,  regardless  where  the  other 
two  UGVs  were.  In  other  situations,  the  target's  escape  routes  might  need  3  or  4  UGVs  for  proper  blockage.  The 
scoring  algorithm  only  took  into  account  whether  the  target's  escape  routes  were  properly  blocked,  not  the  number 
of  robots  contributing  to  the  blockage.  More  specifically,  the  scores  were  based  on  the  percentage  of  escape  routes 
properly  blocked  by  UGVs.  For  example,  two  out  of  two  or  four  out  of  four  would  result  in  the  same  perfect  score 
(i.e.,  100).  One  out  of  two  or  two  out  of  four  would  also  result  in  the  same  score  of  50.  RoboLeader  continuously 
updated  the  calculations  and  visuahsations  (bar  graphs)  until  the  target  was  encapsulated.  The  timing  of  the  target 
encapsulation  ( x  minutes  into  the  scenario  when  the  target  was  encapsulated  for  the  first  time)  was  logged  and 
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(b) 

Figure  2.    (a)  RoboLeader  user  interface  in  the  MIX  Testbed;  (b)  RoboLeader  user  interface  for  route  revisions. 


used  in  the  data  analysis.  The  scores  from  the  horizontal  bars  and  the  vertical  bars  were  not  used  in  the  final  data 
analysis  of  this  study  (they  were  only  used  to  provide  real-time  feedback  to  the  participants  during  experimental 
scenarios). 
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2.2.3.    Surveys  and  tests 

A  demographics  questionnaire  was  administered  at  the  beginning  of  the  training  session.  An  Ishihara  colour  vision 
test  (with  nine  test  plates)  was  administered  via  PowerPoint  presentation  using  the  same  computer/display  system 
for  the  experimental  trials.  The  RoboLeader  user  interface  employed  several  colours  to  display  the  plans  for  the 
robots  and  normal  colour  vision  was  required  in  order  to  effectively  interact  with  the  system.  A  questionnaire  on 
Attentional  Control  (Derryberry  and  Reed  2002)  was  used  to  evaluate  participants'  perceived  attentional  control. 
The  Attentional  Control  survey  consists  of  21  items  and  measures  attention  focus  and  shifting.  Examples  of 
questions  include  'When  I  need  to  concentrate  and  solve  a  problem,  I  have  trouble  focusing  my  attention'  and  T  can 
quickly  switch  from  one  task  to  another.'  The  scale  has  been  shown  to  have  good  internal  rehability  (a  =  0.88).  The 
Cube  Comparison  Test  (Ekstrom  et  al.  1976)  and  the  Spatial  Orientation  Test  (Gugerty  and  Brooks  2004)  were  used 
to  assess  participants'  spatial  abihty.  The  Cube  Comparison  Test  required  participants  to  compare,  in  three- 
minutes,  21  pairs  of  six-sided  cubes  and  determine  if  the  rotated  cubes  were  the  same  or  different.  The  Spatial 
Orientation  Test  was  modelled  after  the  cardinal  direction  test  developed  by  Gugerty  and  Brooks  2004  and  was  a 
computerised  test  consisting  of  a  brief  training  segment  and  32  test  questions.  Both  accuracy  and  response  time  were 
automatically  captured  by  the  programme. 

A  modified  version  of  the  Usabihty  and  Trust  Questionnaire  used  in  Chen  and  Terrence  (2009)  assessed 
participants'  perceived  usability  of  the  RoboLeader  system  as  well  their  trust  in  the  system.  The  questionnaire 
consisted  of  22  questions  on  a  scale  of  1  (strongly  disagree)  to  7  (strongly  agree).  Eleven  of  the  items  were  intended 
to  measure  the  participants'  perceived  utility  and  usabihty  of  RoboLeader.  For  example,  participants  were  asked  to 
evaluate  whether  they  'made  use  of  the  RoboLeader's  recommendations,'  'felt  "lost"  using  the  RoboLeader 
display,'  'rehed  heavily  on  the  RoboLeader  for  the  task,'  and  felt  'the  RoboLeader  display  was  intuitive  and  made  it 
easy  to  determine  how  to  edit  routes.'  The  other  11  items  assessed  participants'  perceived  trust  in  the  RoboLeader 
system  and  included  questions  such  as  'The  RoboLeader  system  is  dependable,'  'I  am  confident  in  the  RoboLeader 
system,'  and  'the  RoboLeader  display  may  have  harmful  effects  on  the  task.'  These  trust-related  items  were  modified 
from  the  'Trust  Between  People  and  Automation'  questionnaire  (Jian  et  al.  2000).  The  'Trust  Between  People  and 
Automation'  questionnaire  has  been  demonstrated  to  have  high  internal  rehability  (Spain  and  Bhss  2008)  and  has 
been  previously  vahdated  by  Spain  et  al.  (2008). 

Participants'  perceived  workload  was  evaluated  using  the  computer-based  version  of  NASA-TLX 
questionnaire  (Hart  and  Staveland  1988).  The  NASA-TLX  is  a  self-reported  questionnaire  of  perceived  demands 
in  six  areas:  mental,  physical,  temporal,  effort  (mental  and  physical),  frustration  and  performance.  Participants 
were  asked  to  evaluate  their  perceived  workload  level  in  these  areas  on  10-point  scales.  They  also  assessed  the 
contribution  (i.e.  weight)  of  each  factor  to  the  perceived  workload  by  comparing  the  15  possible  pairs  of  the  six 
factors.  According  to  Noyes  and  Bruneau  (2007),  computer-based  NASA-TLX  tends  to  generate  higher 
workload  ratings  compared  with  the  traditional  paper-based  survey.  However,  since  the  ratings  were  used  to 
compare  the  workload  levels  across  the  experimental  conditions,  the  elevated  ratings  should  not  affect  these 
comparisons. 

2.5.  Procedure 

After  being  briefed  on  the  purpose  of  the  study  and  signing  the  informed  consent  form,  participants  completed 
the  demographics  questionnaire  and  the  Attentional  Control  survey,  and  were  administered  a  brief  Ishihara 
Colour  Vision  Test  to  ensure  they  had  normal  colour  vision.  After  the  Colour  Vision  Test,  the  participants 
completed  the  two  spatial  abihty  tests.  Participants  then  received  training  and  practiced  on  the  tasks  they  would 
need  to  conduct  during  the  experimental  session.  Training  was  self-paced  and  was  delivered  by  a  video  tutorial 
showing  the  elements  of  the  OCU,  steps  for  completing  various  tasks,  several  hands-on  mini-exercises  for 
practicing  the  steps,  and  exercises  for  performing  the  robotic  control  tasks.  The  training  session  lasted 
approximately  one  hour.  The  participants  had  to  demonstrate  that  they  could  recall  all  the  steps  for  performing 
the  tasks  (including  interpreting  the  visualisation  display)  without  any  help  before  they  started  the  experimental 
session. 

The  experimental  session  lasted  about  1.5  hour  and  began  immediately  after  the  training  session.  Each 
experimental  session  had  four  scenarios  (corresponding  to  the  four  experimental  conditions;  however,  the 
pairing  of  scenarios  and  conditions  was  counterbalanced),  each  lasting  approximately  20  min.  During  the 
scenarios,  participants  used  their  four  ground  robots  (UGVs)  to  pursue  a  primary  moving  target  (a  truck 
traveling  at  about  3  MPH)  while  monitoring  the  streaming  videos  from  the  robots  in  order  to  find  additional 
(secondary)  targets  (insurgents  carrying  weapons)  in  the  mission  environment.  When  the  scenario  for  the 
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Manual  condition  started,  the  participants  put  in  waypoints  for  each  UGV  manually  and  adjusted  the 
waypoints  based  on  the  movement  of  the  primary  target.  In  the  Semi-Autonomous  conditions,  the  participant 
selected  an  end  point/location  for  the  UGV  at  which  time  RoboLeader  provided  an  optimum  solution  (i.e. 
intermediate  waypoints)  with  how  to  reach  the  desired  destination.  With  the  visuahsation  condition,  in  which 
RoboLeader  provided  real-time  feedback  to  the  participants,  the  user  might  consult  the  bar  graphs  as  an 
indicator  of  whether  their  point  selections  were  effective  in  terms  of  synchronisation  of  the  robots  and 
entrapment  of  the  target  or  if  the  plans  needed  revisions.  The  scores  displayed  in  the  visualisation  area  (which 
were  not  included  in  the  final  data  analysis  of  this  study)  were  calculated  based  on  the  RoboLeader' s 
encapsulation  algorithm  and  were  based  on  how  many  of  the  target's  escape  routes  had  a  robot  assigned  to 
them  and  it  was  recalculated  every  time  the  endpoint  of  the  robot  path  changed  and  when  the  target  changed 
directions  (for  more  information  on  the  scoring  algorithm,  see  Chen  et  al.  2011b).  Without  visualisation,  the 
participant  had  to  determine  if  they  were  properly  cornering  the  target  for  capture  (i.e.  entrapping  the  target  by 
at  least  two  robots).  In  the  Fully  Automated  condition,  RoboLeader  provided  the  recommended  endpoints  as 
well  as  intermediate  waypoints  for  each  robot.  The  participant  could  either  accept  or  reject  the  plans.  If  a  plan 
is  rejected,  the  participants  could  still  use  the  rejected  plan  as  a  basis  and  modify  the  waypoints  to  create  a  new 
plan.  RoboLeader's  revisions  were  not  implemented  until  the  participant  accepted  (i.e.  management  by  consent) 
or  modify  them.  In  each  scenario,  there  were  hostile  areas  (indicated  by  red  squares  on  the  map)  that  the 
robots  needed  to  avoid.  The  participants  were  asked  to  finish  the  scenario  if  the  target  was  encapsulated  within 
the  20-minute  limit  (in  order  for  all  the  participants  to  have  equal  amount  of  experiment  time).  In  this  case,  the 
target  would  move  pass  one  of  the  blocking  UGV  and  the  participants  would  have  to  revise  the  plans  to  block 
the  target.  The  simulator  logged  the  amount  of  time  the  participants  spent  on  encapsulating  the  target  (for  the 
first  time)  in  each  scenario.  It  also  logged  whether  the  participants  successfully  encapsulated  the  target  by  the 
end  of  the  scenario  (encapsulation  is  defined  as  at  least  two  of  the  four  robots  blocking  the  road  the  target 
vehicle  was  traversing).  Both  of  these  data  were  analysed  (see  3.1.1).  The  order  of  experimental  conditions  was 
counterbalanced  across  participants. 

The  robots  did  not  have  aided  target  recognition  capability;  therefore,  the  participants  had  to  detect  the 
insurgents  (secondary  targets)  by  themselves.  For  the  insurgent  targets,  participants  used  a  mouse  to  click  the 
Insurgent  button  on  the  interface  and  then  chck  on  the  insurgent  to  'laze'  them  (i.e.  using  the  simulated  laser  range 
finder)  as  soon  as  they  were  detected.  The  'lazed'  insurgent  was  then  displayed  on  the  map.  Additionally,  there  were 
civihans  as  well  as  friendly  Soldiers  in  the  simulated  environment  to  increase  the  visual  noise  present  in  the  target 
detection  tasks.  If  the  participants  accidently  lazed  a  non-insurgent  entity,  the  icon  for  lazed  insurgents  would  not 
appear  on  the  map  and  the  scoring  system  would  not  count  that  as  a  hit. 

Each  scenario  also  contained  five  SA  queries,  which  were  triggered  based  on  time  progression  (e.g.  three 
minute  into  the  scenario).  The  SA  queries  included  questions  such  as  'Use  the  provided  paper  to  identify  which 
route(s)  have  encountered  the  most  Insurgents,'  etc.  When  an  SA  query  was  triggered,  the  OCU  screen  went 
blank,  the  simulation  was  paused,  and  the  SA  query  was  displayed  on  the  screen.  Participants  then  wrote  their 
response  to  the  query  on  an  answer  sheet.  After  participants  responded  to  the  SA  query,  it  was  removed  from  the 
OCU  screen  and  the  simulation  resumed.  During  the  training  session,  participants  were  told  the  SA  queries  they 
would  encounter  in  the  experimental  scenarios  and  had  a  chance  to  practice  answering  those  queries  during  the 
training  exercises. 

There  was  a  two-minute  break  between  the  experimental  scenarios.  Participants  assessed  their  perceived 
workload  using  the  NASA-TLX  as  well  as  their  perceived  usabihty  of  the  RoboLeader  user  interface  using  the 
Usability/Trust  survey  immediately  after  each  experimental  scenario.  The  entire  data  collection  session  lasted  about 
three  hours. 


2,4,    Dependent  measures  and  data  analysis 

The  study  was  a  within- subject  design  with  RoboLeader's  LOAs  as  the  independent  variable  [with  four  levels: 
Manual,  Semi-Autonomous  without  Visuahsation  (Semi-Auto  w/o  Vis),  Semi-Autonomous  with  Visuahsation 
(Semi- Auto  w/  Vis)  and  Fully  Automated  (Auto)].  Dependent  measures  included  the  participants'  performance  of 
encapsulating  the  primary  target  [time  spent  on  encapsulating  the  target  (time  elapsed  until  the  target  was 
encapsulated  for  the  first  time  in  each  scenario)],  the  percentage  of  secondary  targets  (insurgents)  detected,  the 
participants'  SA  of  the  mission  environment  (percentage  of  SA  queries  answered  correctly),  and  the  participants' 
perceived  workload.  A  repeated-measure  analysis  of  variance  with  RoboLeader  as  the  within- subject  factor  was 
used  to  evaluate  the  operator  performance  differences  among  the  four  conditions. 
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3.  Results 

5./.    Operator  task  performance 

3.1.1.    Primary  target  encapsulation  performance 

Table  1  lists  several  measures  relating  to  operator  performance  and  perceived  workload.  The  analysis  revealed  that 
the  LOAs  of  RoboLeader  significantly  affected  the  participants'  performance  of  primary  target  encapsulation  (time 
spent  on  encapsulating  the  moving  target  with  maximum  being  20  minutes  if  target  was  not  encapsulated  before  the 
end  of  the  scenario),  F(3,24)  =  4.29,  p  <  0.05  (Figure  3).  Post  hoc  comparisons  [least  significant  difference  (LSD)] 
showed  that  the  participants  spent  significantly  more  time  on  encapsulating  the  target  in  the  Manual  condition  than 
each  of  the  other  three  RoboLeader-assisted  conditions,  p's  <  0.05.  The  differences  among  the  three  RoboLeader 
conditions  were  not  significant.  We  felt  it  was  important  to  examine  the  latency  data  with  the  20-minute  times 
included  because  a  large  percentage  (57%)  of  the  infrequent  gamers  failed  to  achieve  encapsulation  in  the  Manual 
condition  (see  results  in  the  next  paragraph);  latency  scores  would  possibly  be  biased  by  their  deletion.  Additionally, 
20  minutes  seemed  to  be  a  conservative  estimate  of  the  latency  since  longer  latencies  were  capped  at  20  minutes.  An 
additional  analysis,  however,  was  also  performed  with  the  20-minute  times  removed  from  the  data  (i.e.,  only  the 
times  for  successful  encapsulation  were  included).  The  analysis  showed  that  the  differences  among  the  conditions 
were  not  significant. 

On  average,  63%  of  the  participants  successfully  encapsulated  the  primary  target  in  the  Manual  condition; 
96%  of  them  did  so  in  the  Semi- Autonomous  without  Visuahsation  condition;  86%  in  the  Semi- Autonomous 
with  Visuahsation  condition  and  86%  the  Fully  Automated  condition  (Figure  4).  A  chi-squared  test  showed 
that  the  percentage  of  successful  encapsulation  in  the  Manual  condition  was  significantly  lower  than  the  average 
of  the  other  three  conditions  (i.e.  RoboLeader-assisted  conditions),  (1?  N  =21)  =11.9,  /?<  0.001.  In  the 
Manual  condition,  frequent  gamers  successfully  encapsulated  the  target  70%  of  the  times  and  infrequent  gamers 
did  so  only  43%  of  the  times.  Participants'  spatial  ability  (composite  scores  of  the  two  spatial  tests)  and 
perceived  attentional  control  (Attentional  Control  Survey  scores)  did  not  have  effects  on  their  encapsulation 
performance. 


Table  1.    Mean  operator  task  performance  and  workload  assessments  (standard  deviations  are  presented  in  parentheses). 


Semi-Auto 

Semi-Auto 

Measures 

Manual 

w/o  Visualisqation 

w  Visualisation 

Auto 

Time  spent  on  encapsulating  (min) 

14.19  (5.82) 

8.86  (3.27) 

10.65  (4.67) 

9.70  (4.64) 

%  of  secondary  targets  detected 

46.47  (13.24) 

44.51  (20.51) 

47.71  (20.76) 

43.64  (22.79) 

SA  (%  correct) 

76.2  (13.0) 

68.7  (17.5) 

69.8  (20.2) 

71.8  (15.7) 

WorkloadoNASATLX 

63.9  (10.7) 

55.5  (16.9) 

59.6  (11.4) 

60.2  (17.6) 
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3.1.2.  Secondary  target  detection  performance 

On  average,  the  participants  detected  less  than  half  (44-48%)  of  the  insurgents  in  the  mission  environment. 
However,  there  was  not  a  significant  main  effect  of  RoboLeader's  LOAs  on  participants'  target  detection 
performance.  Across  conditions,  participants  with  higher  spatial  abiUty  (SpA)  detected  significantly  more  targets 
than  did  those  with  lower  spatial  ability,  F(l,23)  =  6.42,  p  <  0.05  (Figure  5). 

3.1.3.  Situation  awareness 

The  participants'  SA  of  the  mission  environment  did  not  differ  among  the  conditions.  Participants'  gaming 
experience  had  an  effect  on  their  SA,  i^(2,24)  =  3.48,  p  <  0.05  (Figure  6).  Post  hoc  comparisons  (LSD)  showed  that, 
across  conditions,  the  frequent  (daily  and  weekly)  gamers  outperformed  infrequent  gamers, />  <  0.05.  There  was  no 
difference  between  daily  and  weekly  gamers. 

5.2.    Perceived  workload 

There  was  not  a  significant  main  effect  of  RoboLeader's  LOAs  on  participants'  perceived  workload  (weighted 
NASA  TLX  composite  scores;  Figure  7).  However,  when  the  workload  assessment  in  the  Manual  condition  was 
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Figure  4.    Percentage  of  participants  with  successful  encapsulation  performance. 
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Figure  5.    Secondary  target  detection  performance  and  spatial  ability  (SpA). 
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Figure  6.    SA  queries. 
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Figure  7.    Perceived  workload. 

compared  against  the  aggregated  workload  assessments  of  the  other  three  conditions  combined,  there  was  a 
significant  difference  between  the  Manual  and  the  RoboLeader-assisted  conditions,  /(26)  =  2.93,  <  0.01. 
Participants  reported  significantly  higher  workload  ratings  in  the  Manual  condition  (M=63.9)  than  they  did  in 
the  other  three  conditions  (M=5SA). 


3,3,    Operators*  interaction  with  the  RoboLeader  user  interface 

Participants'  interaction  with  the  RoboLeader  user  interface  (including  the  visualisation  display)  was  further 
analysed  based  on  their  responses  in  the  Usability/Trust  survey.  The  responses  were  aggregated  into  two  categories: 
perceived  utility  of  RoboLeader  (Ql-10  &  Q22)  and  trust  in  RoboLeader  (Qll-21).  There  were  no  significant 
differences  among  the  three  RoboLeader  conditions  on  either  of  the  aggregate  scores.  In  other  words,  there  were  no 
observable  diff'erences  in  participants'  perceived  utihty  of  RoboLeader  and  their  trust  in  RoboLeader  across  the 
three  conditions.  There  was  also  no  evidence  from  the  survey  data  that  the  added  visualisation  displays  helped 
participants.  Based  on  the  button  presses  captured  from  the  user  interface,  47%  of  the  RoboLeader  route  solutions 
in  the  Full  Auto  condition  were  rejected/modified  by  the  participants.  Participants  with  higher  spatial  abiHty  tend  to 
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modify  RoboLeader's  solutions  more  often  (54%)  than  did  those  with  lower  spatial  ability  (41%),  r  =  0.464, 
p  <  0.01.  However,  there  was  no  evidence  of  a  significant  relationship  between  participants'  rejection  rates  and  their 
task  performance  and/or  workload.  The  rejection  data  were  only  analysed  for  the  Full  Auto  condition,  as  the  other 
three  conditions  required  the  participants  to  put  in  the  endpoints  and/or  intermediate  waypoints  of  the  plans. 


4.  Discussion 

An  intelHgent  agent,  RoboLeader,  was  developed  to  coordinate  a  team  of  ground  robots  and  revise  route  plans  for 
the  robots  based  on  battlefield  intelHgence  (Chen  and  Barnes  2012).  In  the  current  study,  the  capabilities  of 
RoboLeader  were  expanded  to  deal  more  specifically  with  dynamic  re-tasking  requirements  based  on  battlefield 
developments  as  well  as  coordination  between  UAVs  and  UGVs  in  pursuit  of  moving  targets.  The  results  of  our 
human-in-the-loop  simulation  experiment  showed  that  RoboLeader  (the  Auto  and  the  two  Semi-Auto  conditions) 
benefited  participants'  target  encapsulation  performance  compared  with  the  Manual  condition  (when  they  were 
without  assistance  from  RoboLeader).  Participants  were  able  to  encapsulate  the  moving  target  in  shorter  times 
when  they  were  assisted  by  RoboLeader  than  when  they  were  not,  although  the  differences  among  the  three 
RoboLeader-assisted  conditions  did  not  significantly  differ  from  one  another.  Additionally,  participants  successfully 
encapsulated  the  moving  targets  only  63%  of  the  time  in  the  Manual  condition  but  89%  of  the  time  when  they  were 
assisted  by  RoboLeader  (i.e.  the  other  three  conditions  combined).  Interestingly,  only  one  participant  (out  of  28)  in 
the  second  lowest  LOAs  condition  (Semi-Auto  w/o  Vis)  failed  to  encapsulate  the  target  within  the  20-minute  time 
limit.  This  result  suggests  that  the  mixed-initiative  planning  where  the  human  operator  entered  the  end-points  and 
RoboLeader  filled  in  the  intermediate  waypoints  seemed  sufficiently  effective  in  our  simulated  tasking  environment. 
In  the  Manual  condition,  70%  of  those  participants  who  played  video  games  frequently  (daily  or  weekly)  were  able 
to  encapsulate  the  target,  while  only  43%  of  those  infrequent  gamers  (those  who  played  monthly  or  with  even  less 
frequency)  successfully  encapsulated  the  target.  These  results  are  consistent  with  the  findings  of  one  recent  US  Air 
Force  study  (McKinley  et  al  2010)  that  frequent  video  gamers  outperformed  infrequent  gamers  on  robotics  (UAV) 
tasks  and,  in  some  cases,  even  outperformed  experienced  pilots. 

It  is  somewhat  surprising  that  visuaHsation  had  Httle  effect  on  the  Semi- Autonomous  conditions;  possibly  having 
a  separate  visuaHsation  from  the  map  may  have  actually  made  the  task  somewhat  more  difficult.  Follow-on  efforts 
should  investigate  effective  ways  to  integrate  the  map  display  with  the  information  currently  displayed  in  the 
visualisation  portion,  in  order  to  reduce  the  requirement  of  visual  scanning  and  facilitate  more  efficient  utilisation  of 
RoboLeader's  feedback  (Porat  et  al.  2010).  It  is  also  interesting  to  note  that,  in  the  Auto  condition,  47%  of  the 
RoboLeader  route  solutions  (i.e.  route  endpoint  placements)  were  modified  by  the  participants  (especially  those 
with  higher  spatial  ability)  -  although  there  was  not  a  significant  correlation  between  the  participants'  tendency  to 
modify  RoboLeader's  solutions  and  their  encapsulation  performance.  The  relationship  between  participants' 
perceived  attentional  control  abihty  and  their  tendency  to  modify  RoboLeader's  solutions  was  not  clear.  More 
specifically,  participants  with  lower  attentional  control  did  not  have  a  higher  rate  of  accepting  RoboLeader's 
solutions  than  did  those  with  higher  attentional  control.  In  other  words,  those  with  low  attentional  control  did  not 
exhibit  significantly  higher  level  of  complacency  and  trust  in  automation  than  did  those  with  higher  attentional 
control.  The  low  acceptance  rate  of  RoboLeader's  solutions  in  the  Auto  condition  was  similar  to  the  'clumsy 
automation'  described  in  Squire  and  Parasuraman  (2010).  The  main  reason  for  the  high  modification  rates  may  be 
related  to  flaws  in  RoboLeader's  algorithm,  which  should  be  further  improved  in  the  follow-on  research.  It  is  worth 
noting  that  in  the  current  study,  the  speed  of  the  moving  target  was  fairly  slow  (nominally  3  MPH),  which  made  the 
target  encapsulation  task  relatively  easy.  If  the  moving  target  had  traveled  at  a  higher  speed,  the  target 
encapsulation  task  would  have  been  more  difficult  and  the  utihty  of  RoboLeader  and  the  effects  of  operator  spatial 
ability  might  have  been  even  more  pronounced. 

UnHke  previous  findings  (Chen  and  Terrence  2008),  the  LOAs  of  RoboLeader  failed  to  have  any  significant 
effects  on  participants'  concurrent  performance  of  detecting  secondary  targets.  Compared  with  previous 
RoboLeader  experiments,  the  target  detection  rates  of  the  current  study  were  considerably  lower  (44-48%  in  the 
current  study  vs.  61-77%  in  Chen  and  Barnes  2012).  It  is  likely  that  the  continuous  tracking  of  the  moving  target 
drew  the  participants'  visual  attention  away  from  the  secondary  target  detection  task.  Even  when  the  participants 
were  assisted  by  RoboLeader,  they  continued  to  monitor  the  moving  target  and  assessed  the  viability  of  the  plans. 
In  other  words,  as  Parasuraman  and  Manzey  (2010)  noted,  automation  (in  this  case,  RoboLeader)  did  not  simply 
perform  the  task  for  the  participants  -  it  changed  the  nature  of  the  task.  Judging  from  the  secondary  target  detection 
data,  the  visual  attention  required  in  the  RoboLeader-assisted  conditions  was  no  less  than  in  the  Manual  condition. 
It  should  be  noted  that  requiring  the  participants'  active  engagement  in  the  two  Semi-Autonomous  conditions  did 
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not  hurt  performance  -  suggesting  that  requiring  Hmited  active  engagement  may  be  a  compromise  strategy  for 
designing  semi-autonomous  agents  in  order  to  amehorate  automation  bias  and  delegation  authority  issues  (Fern 
and  Shively  2009,  Parasuraman  and  Manzey  2010).  The  dynamics  of  human/agent  roles  need  to  be  delineated 
further,  but  assigning  the  human  a  more  active  role  with  the  agent  mitigating  most  of  the  high  workload  task 
components  (as  opposed  to  having  the  operator  act  as  a  passive  yes/no  supervisor)  may  be  more  effective  in  the  long 
term  (Cosenzo  et  al  2010). 

Consistent  with  prior  studies,  we  found  that  those  participants  with  higher  spatial  abihty  detected  more  targets 
than  did  those  with  lower  spatial  ability  (Chen  et  al.  2008,  Chen  and  Joyner  2009,  Chen  and  Terrence  2008,  2009, 
Chen  2010,  Chen  and  Barnes  2012).  These  results  suggest  that  individuals  with  higher  spatial  abihty  tend  to  exhibit 
more  effective  scanning  performance  and,  therefore,  are  able  to  detect  more  targets  than  do  those  with  lower  spatial 
abihty.  Figure  5  also  shows  different  trends  of  interaction  between  RoboLeader  LOAs  and  operators'  spatial  abihty. 
Those  with  lower  spatial  ability  performed  worse  (detected  fewer  targets)  as  the  LOAs  became  higher;  those  with 
high  spatial  abihty,  on  the  other  hand,  showed  the  opposite  trend.  These  results  suggest  that  if  the  automated  system 
requires  significant  amount  of  visual  attention,  it  may  not  benefit  low  spatial  individuals  as  much  as  high  spatial 
individuals  or  may  require  remedial  training  for  low  spatial  operators.  The  findings  of  the  current  study  also  support 
the  recommendations  by  Lathan  and  Tracey  (2002)  and  two  recent  US  Air  Force  studies  (Chappelle  et  al.  2010a, 
2010b)  that  military  missions  can  benefit  from  selecting  personnel  with  higher  spatial  abihty  to  operate  robotic 
devices.  Training  interventions  that  could  enhance  the  spatial  interpretations  required  to  perform  a  mission  task 
might  also  be  of  benefit  (Baldwin  and  Reagan  2009). 

Notably,  frequent  video  gamers  had  significantly  better  SA  of  the  mission  environments  than  did  infrequent 
gamers.  These  results  are  consistent  with  previous  findings  (Green  and  Baveher  2006,  Chen  and  Barnes  2012) 
suggesting  that  video  game  play  is  associated  with  greater  visual  short-term  memory  and  faster  information 
processing,  which  in  turn,  may  have  contributed  to  game  playing  participants'  superior  SA  in  the  current  study. 
These  results  also  support  the  conclusion  of  a  US  Air  Force  study  (Triplett  2008)  based  on  interviews  of  UAV  pilots 
that  gamers'  superior  SA  may  be  able  to  translate  into  superior  robotics  management  performance. 

The  participants  reported  significantly  higher  workload  when  they  were  in  the  Manual  condition  than  when  they 
were  assisted  by  RoboLeader.  This  finding  is  consistent  with  Chen  and  Terrence  (2008)  and  Chen  and  Joyner  (2009). 
The  higher  workload  associated  with  manual  controls  needs  to  be  taken  into  account  when  designing  the  user 
interfaces  for  the  robotic  management  systems  (Chen  et  al.  2007). 

5.  Conclusions 

In  conclusion,  RoboLeader  demonstrated  significant  utihty  for  enhancing  human  operators'  performance  as  well  as 
reducing  their  workload.  The  results  clearly  demonstrated  the  benefits  of  RoboLeader's  assisting  the  participants  in 
their  route  planning.  Most  importantly,  the  results  suggest  that  agent/human  supervision  of  multiple  robots  permits 
synergy  without  usurping  the  human's  decision  authority  and  those  LOAs  with  less  than  full  autonomy  can  be  as 
effective  as  fully  autonomous  LOAs.  The  Manual  condition,  compared  with  the  other  three  conditions,  was 
associated  with  the  lowest  performance  in  terms  of  speed  of  target  encapsulation,  percentages  of  target 
encapsulation,  and  perceived  workload.  The  results  suggest  that,  compared  with  our  previous  research, 
RoboLeader's  utility  increases  for  more  difficult  scenarios  (e.g.  estimating  mobility  effects  of  moving  targets). 
Furthermore,  it  appears  that  semiautonomous  assistance  of  RoboLeader  was  sufficiently  beneficial  for  the 
encapsulation  task  perhaps  because  it  permitted  the  operator  to  'stay  in  the  loop'  while  reducing  task  difficulty  to  a 
manageable  level  (Endsley  and  Kiris  1995).  Additionally,  displaying  dynamic  re-tasking  information  on  a  map 
display  was  sufficient  to  allow  participants  to  visualise  the  encapsulation  process  without  specialised  displays. 
Finally,  the  current  study  shows  that  individual  differences  such  as  spatial  ability  and  video  gaming  experience  can 
have  a  profound  impact  on  operator's  task  performance  and  can  modulate  the  effectiveness  of  RoboLeader.  Those 
with  higher  spatial  abihty,  compared  with  low  spatial  individuals,  were  better  able  to  detect  secondary  targets  while 
monitoring  and  managing  the  moving  target  encapsulation  task.  Furthermore,  RoboLeader  with  higher  levels  of 
assistance  seemed  to  benefit  the  high  spatial  individuals  in  their  secondary  target  detection  task  more  than  it  helped 
the  low  spatial  individuals  -  likely  due  to  the  amount  of  visual  attention  required  for  even  highly  automated 
RoboLeader.  Frequent  gamers,  compared  with  infrequent  gamers,  had  significantly  better  SA  of  the  mission 
environments.  These  results  may  have  important  imphcations  for  system  design  and  personnel  selection  for  future 
military  programmes  (Triplett  2008,  Chappelle  et  al.  2010a,  2010b,  McKinley  et  al.  2010).  Future  research  can 
investigate  training  interventions  (e.g.  attention  management)  and/or  user  interface  designs  (e.g.  multimodal  cueing 
displays)  to  enhance  robot  operator  performance  in  challenging  tasking  environments  (Chen  et  al.  l^^l,  Dux  et  al. 
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2009,  Chen  et  al.  2011a).  Future  efforts  will  also  examine  the  feasibility  of  implementing  RoboLeader-like  agent  in 
other  mihtary  multi-robot  missions  such  as  building-mapping  and  clearing  and  swarm  control. 
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Objective:  We  evaluate  and  quantify  the  effects  of 
human,  robot,  and  environmental  factors  on  perceived 
trust  in  human-robot  interaction  (HRI). 

Background:  To  date,  reviews  of  trust  in  HRI 
have  been  qualitative  or  descriptive.  Our  quantitative 
review  provides  a  fundamental  empirical  foundation  to 
advance  both  theory  and  practice. 

Method:  Meta-analytic  methods  were  applied  to 
the  available  literature  on  trust  and  HRI.  A  total  of 
29  empirical  studies  were  collected,  of  which  10  met 
the  selection  criteria  for  correlational  analysis  and 
I  I  for  experimental  analysis.  These  studies  provided 
69  correlational  and  47  experimental  effect  sizes. 

Results:  The  overall  correlational  effect  size  for  trust 
was  r  =  +0.26,  with  an  experimental  effect  size  of  d  =  +0.7 1 . 
The  effects  of  human,  robot,  and  environmental  charac- 
teristics were  examined  with  an  especial  evaluation  of 
the  robot  dimensions  of  performance  and  attribute-based 
factors.  The  robot  performance  and  attributes  were 
the  largest  contributors  to  the  development  of  trust  in 
HRI.  Environmental  factors  played  only  a  moderate  role. 

Conclusion:  Factors  related  to  the  robot  itself, 
specifically,  its  performance,  had  the  greatest  current 
association  with  trust,  and  environmental  factors  were 
moderately  associated.  There  was  little  evidence  for 
effects  of  human-related  factors. 

Application:  The  findings  provide  quantitative 
estimates  of  human,  robot,  and  environmental  factors 
influencing  HRI  trust.  Specifically,  the  current  summary 
provides  effect  size  estimates  that  are  useful  in 
establishing  design  and  training  guidelines  with  reference 
to  robot-related  factors  of  HRI  trust.  Furthermore, 
results  indicate  that  improper  trust  calibration  may  be 
mitigated  by  the  manipulation  of  robot  design.  However, 
many  future  research  needs  are  identified. 

Keywords:  trust,  trust  development,  robotics,  human- 
robot  team 
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INTRODUCTION 
Human-Robot  Partnerships 

Robots  are  frequently  used  in  environments 
that  are  unreachable  by  or  are  unsafe  for  human 
beings.  Robotic  operations  include,  among 
others,  planetary  exploration,  search  and  res- 
cue, activities  that  impose  hazardous  levels 
of  workload  on  human  operators,  and  actions 
requiring  complex  tactical  skills  and  informa- 
tion integration  (Hinds,  Roberts,  &  Jones,  2004; 
Parasuraman,  Cosenzo,  &  de  Visser,  2009). 
Robotic  usage  is  penetrating  into  many  diverse 
applicational  realms,  especially  in  the  advanced 
surgical  areas  and  as  assistive  technologies 
for  injured  and  disabled  persons  (Guizzo  & 
Goldstein,  2005;  Heerink,  Krose,  Evers,  &  Wielinga, 
2010;  Tsui  &  Yanco,  2007).  When  they  are  used  in 
military  operations,  robots  are  often  currently 
perceived  as  tools  to  be  manipulated  by  humans 
to  accomplish  specific  discrete  functions  (Chen, 
Barnes,  &  Harper-Sciarini,  2010). 

Yet,  as  robot  capabilities  grow,  the  possibility 
arises  that  they  might  provide  higher  level  fiinc- 
tions  as  full-fledged  team  members.  For  this 
higher  functioning  to  blossom,  however,  effective 
human-robot  partnerships  will  need  to  be  forged 
to  ensure  success  in  dangerous  conflict  situations, 
particularly  because  of  the  increased  stress  and 
cognitive  workload  demands  placed  on  contem- 
porary warfighters  (Hancock  &  Warm,  1989). 

In  future  military  contexts,  warfighters  are 
likely  to  be  mandated  to  interact  with  a  diverse 
inventory  of  robots  on  a  regular  basis,  particularly 
in  dynamic  and  stressful  environments  (Chen  & 
Terrence,  2009).  Already,  robotic  systems  have 
demonstrated  their  usefulness  in  decision  mak- 
ing, communication,  enhancement  of  warfighter 
situation  awareness,  combat  efficiency,  and  reduc- 
ing uncertainty  in  volatile  situations  (Adams, 
Bruyn,  Houde,  &  Angelopoulos,  2003).  However, 
the  assumption  that  introducing  robots  into 
human  teams  will  result  in  better  performance,  as 
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compared  with  when  the  team  or  robot  operates 
independently,  may  not  always  be  justified. 
Although  the  addition  of  robotic  systems  may 
lead  to  improved  team  capabilities,  it  may  also 
create  difficult  challenges  that  need  to  be  over- 
come before  such  hybrid  partnerships  can  work 
more  effectively  (Adams  et  al,  2003). 

Research  continues  to  address  such  chal- 
lenges as  creating  and  validating  metrics  for  the 
evaluation  of  a  wide  spectrum  of  human-robot 
interactions  (HRI)  issues  (Steinfeld  et  al.,  2006); 
designing  human-robot  interfaces  to  facilitate 
interaction,  operator  understanding,  and  situa- 
tion awareness  (Chen  et  al.,  2010;  Chen,  Haas,  & 
Barnes,  2007;  Keyes,  Micire,  Drury,  &  Yanco, 
2010);  translating  qualities  of  good  human 
teammates  into  features  of  the  robot;  and 
encouraging  human  trust  in  robots  (Groom  & 
Nass,  2007),  which  is  perhaps  foremost  among 
these  challenges. 

Human-Robot  Trust 

For  a  human-robot  team  to  accomplish  its 
goal,  humans  must  trust  that  a  robotic  team- 
mate will  protect  the  interests  and  welfare  of 
every  other  individual  on  the  team.  The  level  of 
trust  in  any  robotic  partner  will  be  particularly 
critical  in  high-risk  situations,  such  as  combat 
missions  (Groom  &  Nass,  2007).  Trust  is  impor- 
tant in  these  contexts  because  it  directly  affects  the 
willingness  of  people  to  accept  robot-produced 
information,  follow  robots'  suggestions,  and  thus 
benefit  from  the  advantages  inherent  in  robotic 
systems  (Freedy,  de  Visser,  Weltman,  &  Coeyman, 
2007).  Trust  therefore  very  much  affects  the  deci- 
sions that  humans  make  in  uncertain  or  risky 
environments  (Park,  Jenkins,  &  Jiang,  2008).  For 
example,  the  less  an  individual  trusts  a  robot,  the 
sooner  he  or  she  will  intervene  as  it  progresses 
toward  taskcompletion  (de  Visser,  Parasuraman, 
Freedy,  Freedy,  &  Weltman,  2006;  Steinfeld 
et  al.,  2006). 

However,  some  accounts  from  warfighters  in 
the  field  demonstrate  the  ease  with  which  trust 
can  actually  develop  between  robots  and  humans 
in  stressful  operations.  In  fact,  one  Explosive 
Ordnance  Disposal  unit  named  its  robot  Sgt.  Talon 
and  gave  it  promotions  and  even  Purple  Hearts  for 
its  stellar  bomb  disposal  performance  (Garreau, 
2007).  Other  accounts  by  soldiers,  in  contrast. 


illustrate  the  difficulties  in  trusting  robots  in  these 
situations.  For  instance,  the  SWORD  (Special 
Weapons  Observation  Reconnaissance  Detection) 
system  was  developed  and  deployed  in  Iraq  in 
2007  to  support  combat  operations  (Ogreten, 
Lackey,  &  Nicholson,  2010).  SWORD,  although 
fully  operational,  was  never  used  in  the  field 
because  soldiers  did  not  trust  it  to  function 
appropriately  and  safely  in  dangerous  situations 
because  of  unexpected  movements  caused  by 
technological  malfunctions  (Ogreten  et  al.,  2010). 

As  illustrated  by  these  respective  accounts, 
varying  levels  of  trust  in  robots  currently  exist 
across  the  HRI  domain.  Inappropriate  levels  of 
trust  may  have  negative  consequences,  such  as 
overreliance  on  and  misuse  of  the  system  (in 
cases  of  extremely  high  levels  of  trust)  or  dis- 
use of  the  system  entirely  (in  cases  of  very  low 
levels  of  trust)  (Lee  &  See,  2004;  Parasuraman 
&  Manzey,  2010;  Parasuraman  &  Riley,  1997). 
Both  distrust  and  overreliance  can  undermine 
the  value  of  the  HRI  system.  Trust  also  influ- 
ences neglect  tolerance,  which  is  defined  as  the 
decline  in  semiautonomous  robot  performance 
as  human  attention  is  directed  to  other  tasks  and/ 
or  as  the  task  complexity  increases  (Goodrich, 
Crandall,  &  Stimpson,  2003). 

When  an  individual  places  a  large  amount  of 
trust  in  a  robot  and  does  not  feel  compelled  to 
actively  manage  it,  he  or  she  may  ignore  the 
robot  for  long  periods.  Consequently,  neglect 
tolerance  should  be  appropriately  calibrated  to 
the  capabilities  of  the  robot  and  the  level  of 
human-robot  trust.  Too  much  neglect  can  make 
it  difficult  for  the  individual  to  regain  situation 
awareness  after  redirecting  attention  back 
toward  the  robot.  Too  little  neglect  means  the 
human  operator  is  not  attending  to  his  or  her 
own  personal  tasks,  thus  resulting  in  suboptimal 
system  performance  overall. 

Current  Research 

Trust  in  HRI  is  very  much  related  to  trust  in 
automation  in  general,  which  has  been  studied 
with  respect  to  its  various  performance  influences 
(Chen  et  al.,  2010;  Lee  &  See,  2004;  Parasuraman, 
Sheridan,  &  Wickens,  2008;  Sheridan,  2002). 
Robots  differ  from  most  other  automated  sys- 
tems in  that  they  are  mobile,  are  sometimes  built 
in  a  fashion  that  approximates  human  or  animal 


180 


form,  and  are  often  designed  to  effect  action  at  a 
distance.  Such  differences  could  suggest  that 
human  trust  may  differ  for  robots  versus  other 
forms  of  automation,  although  this  difference 
would  need  to  be  demonstrated  empirically,  and 
few  if  any  such  direct  comparisons  have  been 
conducted  to  date. 

Alternatively,  one  can  begin  with  the  view  that 
human-robot  trust  and  human-automation  trust 
share  similar  fundamental  characteristics  but  allow 
for  the  possibility  for  differences  as  new  evidence 
is  obtained.  Certainly  the  literature  on  human- 
automation  trust  provides  a  fertile  ground  for 
understanding  a  number  of  factors  influencing 
how  humans  trust  other  external  agents.  The 
human-robot  trust  literature  is  more  restricted, 
but  nevertheless,  sufficient  numbers  of  empiri- 
cal studies  have  been  conducted  to  warrant  a 
meta-analysis  to  identify  the  major  factors  cur- 
rently involved. 

Trust  can  be  dynamically  influenced  by  fac- 
tors (or  antecedents)  within  the  robotic  system 
itself,  the  surrounding  operational  environment, 
and  the  nature  and  characteristics  of  the  respec- 
tive human  team  members  (Park  et  al.,  2008). 
Each  of  these  factors  can  play  an  important  role 
in  trust  development.  To  date,  reviews  of  trust  in 
HRI  have  been  qualitative  and  descriptive,  and 
existing  experiments  largely  attempt  to  extrapo- 
late the  optimum  degree  of  trust  for  a  given  out- 
come (e.g.,  team  performance,  reliance  on  the 
robot).  In  doing  so,  a  degree  of  inappropriate 
trust  (i.e.,  excessive  trust  or  too  little  trust)  is 
also  identified  for  each  potential  outcome  of 
HRI,  such  as  over-  or  underreliance  and  poor 
team  collaboration.  The  factors  that  affect  the 
process  by  which  trust  develops  in  any  HRI  sit- 
uation also  need  to  be  considered.  However,  to 
date,  the  existing  body  of  knowledge  has  mostly 
looked  at  the  momentary  state  of  trust  and  not  its 
process  of  development  per  se.  This  latter  evolu- 
tion clearly  awaits  further  investigation. 

Given  the  foregoing  observations,  the  goal  of 
the  current  research  was  to  perform  a  comprehen- 
sive objective  and  quantitative  review  of  identi- 
fied antecedents  of  trust  in  human-robot  teams. 
Meta-analytic  methods  were  applied  to  the 
extant  literature  on  trust  and  HRI  with  the  aim 
of  quantifying  the  effects  of  differing  dimen- 
sions on  human-robot  trust.  Determining  their 


relative  impact  on  trust  will  not  only  provide  an 
indication  of  current  trends  in  the  human-robot 
trust  research,  but  will  also  lead  to  the  identifica- 
tion of  areas  critical  for  future  study.  Consequently, 
our  quantitative  review  contributes  an  empirical 
foundation  upon  which  to  advance  both  theory 
and  practice. 

ANALYTICAL  METHOD 

Sample  of  Studies 

A  literature  search  was  conducted  using  library 
databases  (including PsycINFO,  Psyc ARTICLES, 
PsycBOOKS,  ACM  Digital  Library,  Applied 
Science  and  Technology,  IEEE,  ScienceDirect, 
and  ProQuest  Dissertations  and  Theses).  U.S. 
Army  Research  Laboratory  technical  reports 
were  also  examined  for  relevance.  In  addition, 
we  used  a  number  of  web-based  search  engines, 
for  example,  Google  and  its  derivative  Google 
Scholar,  to  seek  further  references  not  discovered 
by  the  initial  formal  scan.  The  primary  search 
terms  included  human-robot  interaction,  robot, 
and  trust.  After  the  initial  listing  of  articles  was 
obtained,  reference  lists  were  checked  to  deter- 
mine whether  any  other  related  studies  could  be 
included.  In  a  concurrent  process,  subject  matter 
experts  (SMEs)  were  consulted  for  reference 
to  articles  that  had  not  been  identified  by  the 
prior,  formal  search  procedure.  SMEs  were  drawn 
from  military,  industry,  and  academia  on  the 
basis  of  their  willingness  to  participate  and  their 
availability. 

Following  this  initial  procedure,  we  examined 
the  collected  literature  and  identified  potential 
factors  associated  with  the  development  of  trust. 
SMEs  also  provided  guidance  in  identifying  fac- 
tors influencing  trust  in  human-robot  relation- 
ships. On  the  basis  of  these  identified  factors,  we 
conducted  specific  searches  in  the  aforemen- 
tioned databases  using  the  primary  search  terms 
robot  and  trust  combined  with  these  secondary 
terms:  prior  experience,  attentional  capacity, 
expertise,  competency,  personality,  attitudes, 
propensity  to  trust,  self-confidence,  false  alarm, 
failure  rate,  automation,  anthropomorphism,  pre- 
dictability, proximity,  robot  personality,  multi- 
tasking, workload,  task  load,  culture,  shared 
mental  models,  and  situation  awareness.  When 
these  elicitation  processes  no  longer  yielded 


181 


new  citations,  we  compiled  the  final  listing  of 
articles.  This  process  resulted  in  29  empirical 
articles,  reports,  dissertations,  and  conference 
proceedings  published  between  1996  and  2010. 
Of  these,  1 0  papers  containing  69  correlational 
effect  sizes  and  1 1  papers  containing  47  experi- 
mental effect  sizes  met  selection  criteria  for 
inclusion. 

Criteria  for  Study  Inclusion 

All  studies  were  inspected  to  ensure  that  they 
fulfilled  the  following  four  criteria  for  inclusion 
in  the  meta-analysis:  (a)  Each  study  had  to  report 
an  empirical  examination  of  trust  in  which  trust 
was  a  directly  measured  outcome  of  an  exper- 
imental manipulation.  Studies  in  which  trust 
served  as  the  experimental  manipulation  were 
excluded,  (b)  The  empirical  examination  of  trust 
was  directed  toward  a  robot.  Thus,  for  instance, 
studies  on  human-automation  trust  focusing  on  a 
decision  aid  were  excluded  because  the  empha- 
sis of  such  research  is  on  the  decision  aid  and  not 
a  robot,  which  as  discussed  earlier  can  differ  in 
many  ways  from  automated  systems  in  terms  of 
such  factors  as  mobility,  sensor  and  effector  capa- 
bilities, and  so  on.  (c)  The  study  had  to  incorpo- 
rate human  participants  who  either  viewed  or 
participated  directly  in  interactions  with  a  robot 
through  physical,  virtual,  or  augmented  means, 
(d)  Each  study  had  to  include  sufficient  infor- 
mation to  determine  effect  size  estimates. 

Papers  and  literature  meeting  these  criteria 
are  identified  in  the  reference  listing  in  the  pres- 
ent article  by  an  asterisk  appearing  in  front  of 
the  first  author's  name  (American  Psychological 
Association,  2001).  It  is  important  to  note  that 
rejecting  primary  studies  in  a  meta-analysis  is  a 
common  occurrence  and  is  necessary  to  ensure 
meaningful  results  when  combining  effect  sizes 
across  studies. 

Identification  of  Possible 
Antecedents  of  Trust 

Studies  included  in  the  meta-analysis  were 
classified  into  three  broad  categories  according  to 
the  experimental  manipulation:  robot-related  fac- 
tors (including  performance-based  and  attribute- 
based  factors),  human-related  factors  (including 
ability-based  and  human  characteristic  factors), 
and  environment-related  factors  affecting  trust 


(including  team  collaboration  and  task-based 
factors).  These  differentiates  enabled  a  quanti- 
tative review  of  the  predictive  strength  of  these 
respective  trust  factors  in  human-robot  teams. 
See  Figure  1  for  factors  identified  as  potential 
antecedents  of  human-robot  trust  on  the  basis  of 
the  literature  review  and  SME  guidance. 

The  Calculation  of  Effect  Size 

A  meta-analytic  approach  was  used  to  evalu- 
ate the  data  collected  to  determine  the  pattern  of 
findings  in  the  contemporary  body  of  human- 
robot  trust  research.  First,  each  study's  effect 
size  was  calculated  using  standard  formulas  (see 
Hedges  &  Olkin,  1985;  Hunter  &  Schmidt, 
2004;  Morris  &  Deshon,  2002).  Studies  included 
in  effect  size  calculation  contained  both  correla- 
tional and  group  design  data;  therefore  the  use 
of  multiple  meta-analytic  methods  (correlation 
and  Cohen's  d)  was  necessary.  The  correlational 
effects  represent  an  association  between  trust 
and  the  given  factor.  Cohen's  d  indicates  the 
standard  difference  between  two  means  in  stan- 
dard deviation  units.  From  these,  we  can  gather 
correlational  and  causal  inferences  between 
trust  and  any  given  factor.  Through  both  types 
of  meta-analytic  effects,  the  more  positive  the 
effect,  the  more  trust.  Findings  were  interpreted 
with  the  use  of  Cohen's  (1988)  established  ranges 
for  small  {d  <  .20;  r  <  .10),  medium  {d  =  .50; 
r  =  .25),  and  large  (d  >  .80;  r  >  .40)  effect  sizes. 

Variance  Estimates 

Several  variance  estimates  were  calculated. 
First,  variability  of  the  effect  sizes  themselves 
(s^^)  and  variability  attributable  to  sampling 
error  (^^)  were  estimated.  Next,  these  two  val- 
ues were  used  to  compute  the  residual  variance 
(^^5).  A  large  (s^d)  is  an  indication  that  the  effect 
sizes  may  be  heterogeneous  and  therefore  one  or 
more  variables  are  likely  to  be  moderating  the 
magnitude  of  that  particular  effect.  A  final  check 
for  homogeneity  of  variance  (s^  ^|)  was  calcu- 
lated (proportion  of  total  variance  accounted  for 
by  sampling  error).  Hunter  and  Schmidt  (2004) 
suggest  that  an  outcome  of  0.75  or  greater  sug- 
gests that  the  remaining  variance  is  attributable 
to  a  variable  that  could  not  be  controlled  for  and 
represents  homogeneity  of  variance.  However, 
large  residual  variance  and  small  homogeneity 
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Figure  1.  Factors  of  trust  development  in  human-robot  interaction.  These  factors  were  identified  a  priori  via 
Hterature  review  and  subject  matter  expert  guidance.  Factors  included  in  the  correlational  analysis  are  starred 
(*).  Factors  included  in  the  experimental  analysis  are  crossed  (+). 


of  variance  may  be  seen  because  of  a  small 
number  of  sample  studies,  as  is  evident  in  some 
of  the  following  results  (see  Lipsey  &  Wilson, 
2001,  for  an  in-depth  examination  of  the  vari- 
ous strengths  and  weaknesses  relating  to  meta- 
analytic  procedures). 

RESULTS 

Overall  Outcome  Effects 

Correlational  analysis.  For  the  10  studies 
reporting  correlational  data,  the  present  meta- 
analytic  results  indicated  that  there  was  a  mod- 
erate global  effect  between  trust  and  all  factors 
influencing  HRI  (f  =  +0.26;  see  Table  1).  That 
the  identified  confidence  interval  does  not 
include  zero  confirms  that  this  identified  rela- 
tionship is  consistent  and  substantive.  The 


subsidiary  analysis  between  trust  and  human, 
robot,  and  environmental  factors  individually 
indicated  only  small  effects  for  the  human 
dimensions  (f  =  +0.09)  and  also  the  environ- 
mental characteristics  (f  =  +0.11),  and  because 
the  confidence  intervals  for  human  and  envi- 
ronmental factors  included  zero,  our  current 
state  of  knowledge  suggests  that  the  human  and 
the  environment  are  not  strongly  associated  with 
trust  development  in  HRI  at  this  point  in  time. 
We  should,  however,  emphasize  that  these 
results  derive  from  only  a  limited  number  of 
studies  and  thus  may  change  with  future 
evaluations. 

Robot-related  characteristics  were  found  to 
be  moderately  associated  with  trust  in  HRI  (f  = 
+0.24),  in  line  with  the  level  of  the  global  effect. 
Robot  influences  were  able  to  be  parsed  into  two 
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TABLE  1:  Formal  Human-Robot  Trust  Meta-Analysis  Results  With  Correlational  Data:  Global,  Trust 
Factors,  and  Robot  Factors 


Category 

k 

r 

^  e'^  p 

95%  CI 

n 

Global 

10 

+.26 

AAr 

.01 

.13 

.05 

+.21  <6<+.31 

1,228 

Trust  factors 

Robot 

8 

+.24 

.21 

.01 

.20 

.05 

+.16<6<+.31 

882 

Human 

7 

+.09 

.14 

.02 

.13 

.11 

.00<5<+.19 

727 

Environment 

4 

+.11 

.11 

.01 

.10 

.08 

+.02  <  6  <  +.20 

645 

Robot  factors 

Attribute 

5 

+.03 

.08 

.02 

.07 

.22 

-.09<6<+.15 

686 

Performance 

5 

+.34 

.43 

.01 

.42 

.03 

+.25  <  6  <  +.43 

607 

Note,  k  =  number  of  studies;  n  =  sample  size;    estimates  the  variability  of  the  effect  sizes  themselves;  estimates 
the  variability  attributable  to  sampling  error;     is  an  estimate  of  the  residual  variance;  s^/sp  is  a  calculation  of 
homogeneity  of  variance;  CI  =  confidence  interval. 


subcategories:  robot  performance-based  factors 
(e.g.,  reliability,  false  alarm  rate,  failure  rate)  and 
attribute-based  factors  (e.g.,  proximity,  robot  per- 
sonality, and  anthropomorphism).  With  respect 
to  the  influence  of  the  robot,  it  was  determined 
that  performance  factors  were  more  strongly 
associated  (f  =  +0.34)  with  trust  development 
and  maintenance.  However,  in  contrast,  robot 
attributes  had  only  a  relatively  small  associated 
role  (f  =  +0.03).  Such  potential  influences  for 
human-  and  environmental-related  factors  were 
not  examined,  as  there  were  insufficient  samples 
at  the  present  juncture  to  run  acceptable  quanti- 
tative meta-analytic  procedures. 

Experimental  analysis.  In  contrast  to  studies 
that  presented  correlational  data,  experimental 
studies  reported  group  differences.  Therefore, 
for  this  latter  meta-analysis,  we  used  Cohen's  d. 
The  results  for  the  meta-analytic  approach  with 
the  use  of  Cohen's  d  produced  a  similar  pattem  to 
that  for  the  correlational  studies.  These  results, 
shown  in  Table  2,  indicated  there  was  a  large 
global  effect  conceming  trust  and  HRI  {d = +0.7 1). 
As  the  confidence  interval  excluded  zero,  we  can 
assume  this  is  a  substantive  and  consistently 
large  effect.  The  subdivision  of  this  global  effect 
into  robot,  human,  and  environmental  charac- 
teristics indicated  that  the  robot  (d  =  +0.67)  had 
the  greatest  effect.  There  was  a  moderate  effect 
for  environmental  factors  (d  =  +0.47)  but  only 
very  small  effects  for  human  factors  (d  =  -0.02). 
Robot  factors  were  again  parsed  into  the  two 


categories  of  attributes  and  performance. 
Robot  performance  factors  (d  =  +0.71)  were  the 
largest  identifiable  influences  on  HRI  trust, 
whereas  robot  attributes  (d  =  +0.47)  had  a 
smaller  but  still  sizeable  influence  on  trust 
development. 

We  should,  however,  point  out  that  the  perfor- 
mance factors  are  based  on  two  studies,  which 
may  bring  into  question  the  stability  of  the  effect. 
However,  each  study  has  a  sizable  effect  sup- 
porting this  found  effect.  The  attribute  factors 
are  based  on  eight  studies,  pointing  to  stronger 
stability  of  the  effect.  Specific  influential  effects 
for  human-  and  environment-related  factors  were 
not  examined,  as  there  were  insufficient  data  to 
run  the  meta-analysis.  In  all  of  the  aforemen- 
tioned categories  in  which  there  were  sufficient 
data  to  identify  effects,  none  of  the  confidence 
intervals  for  the  experimental  work  included 
zero.  Therefore,  we  can  have  a  degree  of  confi- 
dence that  these  are  each  consistent  and  real 
effects. 

DISCUSSION 

Trust  is  a  crucial  dimension  in  maintaining 
effective  relationships  with  robots.  The  pres- 
ence, growth,  erosion,  and  extinction  of  trust 
have  powerful  and  lasting  effects  on  how  each 
member  of  any  shared  relationship  behaves  and 
will  behave  in  the  future.  Currently,  we  see  tech- 
nology (and  the  present  panoply  of  robots)  as 
largely  insensate  and  without  individual  motive 
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TABLE  2:  Formal  Human-Robot  Trust  Meta-Analysis  Results  With  Cohen's  d:  Global,  Trust  Factors,  and 
Robot  Factors 


Category 


s2 


95%  CI 


Global 

Trust  factors 
Robot 
Human 


11  +.71 


.26 


.09   .16     .36      +.53<5<+.89  1,567 


.07  .08  .48  +.48<6<+.85  1,119 
202 


8  +.67  .15 
2  -.02  g  =  +.01  (Kidd,  2003);  g  =  -.88 

(Scopelliti,  Giuliani,  &  Fornana,  2005) 
Environment     5  +.47  .21  .07    .13     .36      +.23<6<+.71  609 


Robot  factors 
Attribute  8  +.47  .25 

Performance     2  +.71  g  =  +.71  (Ross,  2008);  g  =  +.74 
(Tsui,  Desai,  &  Yanco,  2010) 


.07  .19  .27  +.28<6<+.65  1,119 
554 


Note.  k=  number  of  studies;  n  =  sample  size;     estimates  the  variability  of  the  effect  sizes  themselves; 

Sg  estimates  the  variability  attributable  to  sampling  error;  s^6  is  an  estimate  of  the  residual  variance;  s^/s^  is  a 

calculation  of  homogeneity  of  variance;  CI  =  confidence  interval. 


force.  Although  we  are  often  frustrated  with  tech- 
nological shortcomings  and  failures  and  express 
our  frustration  accordingly,  at  heart,  we  know  we 
are  dealing  with  the  residual  effects  of  a  remote 
human  designer.  However,  we  stand  on  the  verge 
of  a  sufficiently  impactful  change  that  our  attri- 
bution of  intentionality  to  all  technology  will  soon 
be  justified  (and  see  Epley,  Waytz,  &  Cacioppo, 
2007;  Moravec,  1988).  At  this  juncture,  the  issue 
of  trust  in  technological  systems  will  be  as  influ- 
ential on  social  development  as  it  is  in  our  own 
human-human  relationships. 

Trust  is  only  one  of  a  number  of  critical  ele- 
ments essential  to  human-robot  collaboration,  but 
it  continues  to  be  a  growing  concern  as  robots 
advance  in  their  functionality.  This  is  especially 
the  case  in  military  and  emergency  contexts  in 
which  a  warfighter's  or  an  operator's  own  life 
and  the  lives  and  safety  of  others  depend  on 
successful  interaction.  The  present  research 
represents  one  of  the  first  systematic  efforts 
to  quantify  effects  concerning  human  trust  in 
robots.  Our  results  reveal  that  robot  character- 
istics, and  in  particular,  performance-based 
factors,  are  the  largest  current  influence  on 
perceived  trust  in  HRL  These  findings  imply 
that  manipulating  different  aspects  of  the 
robot's  performance  influences  trust  the  most. 
This  finding  is  central  to  the  consideration  of 
coming  robot  design.  Trends  in  the  literature 


indicate  that  higher  trust  is  associated  with  higher 
reliability  (for  example,  see  Ross,  2008). 
Furthermore,  the  type,  size,  proximity,  and 
behavior  of  the  robot  also  affect  trust  (for  exam- 
ples, see  Bainbridge,  Hart,  Kim,  &  Scassellati, 
2008;  Tsui,  Desai,  &  Yanco,  2010).  Taking  such 
factors  into  consideration  can  have  meaningful 
influence  on  future  robot  design  and  associated 
human-robot  interaction,  although  further  research 
is  still  needed  to  develop  specific  design  heuristics. 

Environmental  factors  were  also  found  to 
be  moderately  influential  on  trust  development. 
Team  collaboration  characteristics  and  tasking 
factors,  as  identified  by  SMEs  and  in  the  literature 
itself,  were  included  in  this  analysis.  However,  fur- 
ther specification  of  team-related  and  task-related 
effects  could  not  be  drawn  because  of  the  insuf- 
ficient number  of  empirically  codable  studies. 
Limited  evidence  for  human-related  factors  was 
found.  The  present  findings,  however,  should 
not  be  taken  to  imply  that  human  characteristics 
in  HRI  are  not  necessarily  important.  Rather,  the 
small  number  of  studies  found  in  this  area 
suggests  a  strong  need  for  future  experimental 
efforts  on  human-related,  as  well  as  environment- 
related,  factors. 

Although  human-automation  interaction  in 
general  has  been  researched  in  more  depth 
(Dzindolet,  Peterson,  Pomranky,  Pierce,  &  Beck, 
2003;  Lee  &  See,  2004;  Madhavan  &  Wiegmann, 
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2007;  Sheridan,  2002;  Sheridan  &  Parasuraman, 
2006),  sparse  empirical  research  has  been  con- 
ducted in  a  number  of  specific  and  important 
areas  associated  with  human-robot  trust.  For 
instance,  as  noted,  there  is  a  dearth  of  studies  on 
the  human-related  characteristics,  including  prior 
level  of  operational  experience,  attentional 
capability,  the  amount  of  training  received,  self- 
confidence,  the  propensity  to  trust,  existing  atti- 
tudes toward  robots,  personality  traits,  operator 
workload,  situation  awareness,  and  other  individ- 
ual difference  factors  (see  Hancock,  Hancock,  & 
Warm,  2009).  Gaps  in  the  understanding  of  the 
various  environmental  characteristics  include 
culture  (of  the  team,  the  individual,  and  the  ambi- 
ent environment),  shared  mental  models,  multi- 
tasking requirements,  task  complexity,  and  task 
type.  We  also  have  limited  empirical  evidence 
on  the  effects  of  robot  "false"  alarms.  Resolution 
in  these  areas  is  crucial  to  provide  an  increasing 
depth  of  understanding  on  trust  in  HRI. 

Our  meta-analytic  findings  have  implications 
for  both  research  and  practice.  In  terms  of  research, 
as  we  build  functional  models  of  HRI,  we  will 
need  to  understand  and  quantify  the  various 
influences  and  derive  information  on  factors  we 
have  shown  that,  to  date,  are  completely  miss- 
ing. Without  a  larger  and  active  empirical  attack, 
our  knowledge  will  remain  precarious  and  based 
often  on  either  anecdotal  or  engineering-centered 
case  studies.  With  regard  to  practical  implica- 
tions, the  major  lesson  learned  is  that  a  robot's 
performance  and  attributes  should  be  consid- 
ered the  primary  drivers  of  trust.  Understanding 
exactly  how  these  factors  affect  the  develop- 
ment of  trust  will  be  critical  for  trust  calibration. 
For  example,  we  are  aware  of,  and  have  cited  one 
instance  of,  a  number  of  occasions  in  the  military 
in  which  robots  have  looked  to  be  deployed,  but 
because  of  the  intrinsic  trust  question,  they 
have  never  been  taken  "out  of  the  box"  (often 
because  of  a  bad  reputation  preceding  the  sys- 
tem or  its  perceived  complexity  of  operation). 
Consequently,  if  the  perceived  risk  of  using  the 
robot  exceeds  its  perceived  benefit,  practical 
operators  almost  always  eschew  its  use.  Hence, 
training  that  encourages  trust  in  specific  robots 
is  necessary  from  the  point  of  design  inception 
on,  until  its  eventual  field  use. 


The  implications  of  this  research  can  also  be 
applied  to  trust  in  HRI  in  a  number  of  critical 
areas  outside  of  the  military.  These  include,  espe- 
cially, considerations  in  the  medical  and  health 
care  arenas.  Assistive  robotic  technologies  are 
also  being  developed  and  tested  for  mobility  pur- 
poses, rehabilitation,  and  of  course,  surgical  aids 
(Tsui  &  Yanco,  2007).  Social  robots  for  domestic 
use  are  also  being  designed  to  help  people  with 
cognitive  and  physical  challenges  maintain  a  high 
quality  of  life  (Heerink,  et  al.,  2010).  For  exam- 
ple, the  Care-O-Bot  II  is  a  robotic  home  assistant 
that  was  created  to  support  the  needs  of  elderly 
persons  in  private  homes,  allowing  individuals 
greater  independence  without  the  necessary 
presence  of  a  human  caregiver  (Graf,  Hans,  & 
Schraft,  2004). 

Other  robotic  devices,  such  as  wheelchairs 
(Yanco,  2001)  and  exoskeletons  (e.g.,  robotic 
arms,  legs;  Guizzo  &  Goldstein,  2005)  can 
benefit  disabled  individuals  by  enhancing  their 
remaining  physical  capabilities.  Furthermore, 
robots  are  capable  of  telepresence  in  life- 
threatening  situations  when  medical  profession- 
als cannot  be  physically  present  as  well  as  in 
cases  involving  high  risk,  such  as  search-and- 
rescue  operations  (Murphy,  Riddle,  &  Rasmussen, 
2004)  and  repairs  in  contaminated  conditions,  as 
the  recent  Japanese  nuclear  power  plant  experi- 
ence has  shown  (Glionna  &  Nagano,  201 1).  Given 
our  current  meta-analytic  findings,  robot  perfor- 
mance factors  in  these  domains  are  also  therefore 
most  critical  to  consider.  Regardless  of  context, 
a  user  must  trust  the  robot  to  enable  effective 
interaction. 

Future  Research 

The  type  of  trust  measure  used  is  relevant 
to  the  present  conclusions.  Our  meta-analysis 
found  that  current  trust  in  HRI  is  derived  almost 
exclusively  via  subjective  response,  measured 
one  time  after  a  specific  interaction.  However, 
physiological  indicators,  such  as  oxytocin-related 
measures,  and  objective  measures,  such  as  trust 
games  that  assess  actual  investment  behavior, 
are  used  frequently  in  the  human-interpersonal 
trust  literature  (for  examples,  see  Chang,  Doll, 
van  't  Wout,  Frank,  &  Sanfey,  2010;  Keri,  Kiss, 
&  Kelemen,  2009).  These  measures  should  be 
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explored  in  the  context  of  human-robot  trust  to 
augment  the  present  perceptual  assessments  and 
identify  potential  inconsistencies  between  such 
measures.  Discrepancies  between  an  individual's 
self-report  (i.e.,  perception)  and  his  or  her  behavior 
(i.e.,  observable  reaction)  is  an  issue  that  has  been 
a  topic  of  concern  in  psychology,  and  especially 
applied  psychology,  for  a  number  of  decades  (see 
Hancock,  1996;  Natsoulas,  1967).  An  individual 
can  report  that  he  (or  she)  will  trust  a  robot,  but 
existing  research  leads  us  to  believe  that  this 
statement-action  relationship  is  not  always  per- 
fect (Chen  &  Terrence,  2009).  Therefore,  empir- 
ical research  that  includes  both  subjective  and 
objective  measurements  can  provide  a  more 
complete  portraiture  of  the  genesis  and  persis- 
tence of  trust. 

A  comparison  of  perceptions  and  actual  robot 
capabilities  is  also  needed.  Each  person  in  a  team 
can  have  differing  perceptions  of  the  intent,  per- 
formance, and  actions  of  a  robotic  entity,  but 
indeed,  these  perceptions  may  not  all  match  the 
true  capabilities  of  the  robot.  These  differences 
in  perception  may  be  mitigated  to  an  extent  by 
employing  training  methods  that  adequately  pre- 
pare an  individual  for  the  coming  interaction.  In 
summary,  numerous  avenues  of  research  need  to 
be  pursued  to  fully  comprehend  the  role  that  trust 
plays  in  HRI  as  well  as  the  factors  that  influence 
trust  in  robots  themselves  as  stand-alone  entities. 
Even  so,  our  current  findings  indicate  that  cur- 
rently, the  most  important  element  of  trust  is  robot 
related.  Fortunately,  these  factors  (e.g.,  robot  per- 
formance, robot  attributes)  can  be  directly  manip- 
ulated by  designers  (with  the  constraints  of 
technological  capabilities).  In  this  way,  we  are 
able  to  predict  to  some  degree  the  development 
of  trust  in  human-robot  partnerships  in  existing 
systems. 
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KEY  POINTS 

•  The  meta-analytic  procedures  included  10  refer- 
ences for  correlational  analysis  (yielding  69  cor- 
relational effect  sizes)  and  11  for  experimental 
analysis  (yielding  47  experimental  effect  sizes). 

•  Robot  characteristics,  and  in  particular,  robot  per- 
formance, were  found  to  be  the  most  important 
influences  of  trust  development. 

•  Environmental  factors  moderately  influenced 
trust,  whereas  little  evidence  was  found  for  the 
effect  of  human  characteristics  on  trust  in  human- 
robot  interaction  (HRI). 

•  Although  human  dimensions  played  a  small  role 
in  trust  development,  the  lack  of  findings  may 
be  attributable  to  insufficient  empirical  data 
(suggesting  limitations  in  the  research),  which 
should  be  addressed  in  future  experimentation. 

•  The  current  summary  of  findings  emphasizes  the 
importance  of  focusing  on  robot-related  factors  in 
design  and  training  guidelines  for  HRI. 
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